# Making Releases ```{contents} :local: ``` ## Introduction ```{note} This document is about releasing the main Dataverse app (). See {doc}`making-library-releases` for how to release our various libraries. Other projects have their own release documentation. ``` ```{note} Below you'll see branches like "develop" and "master" mentioned. For more on our branching strategy, see {doc}`version-control`. ``` Dataverse releases are time-based as opposed to being feature-based. That is, we announce an approximate release date in advance (e.g. for [6.8](https://groups.google.com/g/dataverse-community/c/Y0G9mw4raLU/m/om8vjjVAAQAJ)) and try to hit that deadline. If features we're working on aren't ready yet, the train will leave the station without them. We release quarterly. We also announce "last call" dates for both community pull requests and those made by core developers. If you are part of the community and have made a pull request, you have until this date to ask the team to add the upcoming milestone to your pull request. The same goes for core developers. This is not a guarantee that these pull requests will be reviewed, tested, QA'ed and merged before {ref}`code freeze `, but we'll try. ## Regular or Hotfix? Early on, make sure it's clear what type of release this is. The steps below describe making both regular releases and hotfix releases. - regular - e.g. 6.5 (minor) - e.g. 7.0 (major) - hotfix - e.g. 6.4.1 (patch) - e.g. 7.0.1 (patch) ## Ensure Issues Have Been Created We have a "create release issues" script at that should be run a week or so before code freeze. A parent issue is created (see the [6.12 example](https://github.com/IQSS/dataverse-pm/issues/574)) and a number of sub-issues. For each issue that is created by the script there is likely a corresponding step in this document that has "dedicated" label on it like this: Dedicated Issue  There are a variety of reasons why a step might deserve its own dedicated issue: - The step can be done by a team member other than the person doing the release. - Stakeholders might be interested in the status of a step (e.g. has the release been deployed to the demo site). Steps don't get their own dedicated issue if it would be confusing to have multiple people involved. Too many cooks in the kitchen, as they say. Also, some steps are so small the overhead of an issue isn't worth it. ## Announce the Timeline for the Next Release Dedicated Issue  For the next release in our list of [milestones](https://github.com/IQSS/dataverse/milestones), pass the release date to {download}`generate_release_dates.py <../../../../scripts/dev/release-dates/generate_release_dates.py>`. Put these dates on the milestone and accounce them. See examples from the [Google Group](https://groups.google.com/g/dataverse-community/c/kKh4YUBzU9I/m/7wF1048PCgAJ) and [Zulip](https://dataverse.zulipchat.com/#narrow/channel/375707-community/topic/Release.206.2E12.20Timeline/near/615705804). ## Check If Payara, Solr, or PostgreSQL Should Be Updated Some upgrades are easy. Others require significant code changes, especially for Payara and Solr. Try to strike a balance between giving the team enough time to make code changes and being up-to-date with the latest release. Check to see if there has been a new Payara release. If so, for any security vulnerabilities, try to figure out (with the team's help) if they are serious enough that we should update Payara as part of the release. If so, create an issue, give it the next milestone, and put it in "ready for triage" so the team can size it. Do the same for Solr by checking . That page also shows when versions go EOL, which is good to check. For PostgreSQL, sysadmins running Dataverse are usually not prevented from upgrading to the latest minor PostgreSQL version. Often, they receive updates through a Linux package manager. That said, it's good to check from time to time to see when the major version is going EOL. ## Push Back Milestones on Pull Requests That Missed the Train As the code freeze date approaches, work with the team to decided which pull requests won't make the cut, and bump them to the next release. Don't worry. There will be [another train](https://github.com/IQSS/dataverse/milestones). 🚂 (declare-code-freeze)= ## Declare a Code Freeze The code freeze date is announced well in advance. When we declare a code freeze, we mean: - No additional features will be merged until the freeze is lifted. - Bug fixes will only be merged if they relate to the upcoming release in some way, such as fixes for regressions or performance problems in that release. - Pull requests that directly affect the release, such as bumping the version, will be merged, of course. The benefits of the code freeze are: - The team can focus on getting the release out together. - Regression and performance testing can happen on code that isn't changing. - The release notes can be written without having to worry about new features (and their release note snippets) being merged in. In short, the steps described below become easier under a code freeze. (write-release-notes)= ## Write Release Notes Dedicated Issue  Developers express the need for an addition to release notes by creating a "release note snippet" in `/doc/release-notes` containing the name of the issue they're working on. The name of the branch could be used for the filename with ".md" appended (release notes are written in Markdown) such as `5053-apis-custom-homepage.md`. See {ref}`writing-release-note-snippets` for how this is described for contributors. The task at or near release time is to collect these snippets into a single file. - Find the issue in GitHub that tracks the work of creating release notes for the upcoming release. - Create a branch, add a .md file for the release (ex. 6.10.1 Release Notes) in `/doc/release-notes` and write the release notes, making sure to pull content from the release note snippets mentioned above. - We don't want readers to reach a dead end. Don't just write, "This or that bug was fixed." Always include at least the pull request number so that the reader can click something to learn more. Write somethine like "This or that bug was fixed. See #1234." For features, include a link to the guides as well. - Delete (`git rm`) the release note snippets as the content is added to the main release notes file. - Include instructions describing the steps required to upgrade the application from the previous version. These must be customized for release numbers and special circumstances such as changes to metadata blocks and infrastructure. These instructions are required for the next steps (deploying to various environments) so try to prioritize them over finding just the right words in release highlights (which you can do later). - We usually include a "Security Updates" section due to dependencies we've updated. Under that section, give credit to any security researchers who have reported vulnerabilities. In the [6.11 release notes](https://github.com/IQSS/dataverse/releases/tag/v6.11), for example, we wrote "We would like to thank [person1], [person2], and [person3] for notifying us about vulnerabilities that were fixed in this release." See {ref}`security-researcher-credit` and {ref}`reporting-security-issues`. - Make a pull request. Here's an example: - Note that we won't merge the release notes until after we have confirmed that the upgrade instructions are valid by performing a couple upgrades. For a hotfix, don't worry about release notes yet. ## Build Release Candidate Dedicated Issue  Go to click "run workflow". For a regular release, make sure the branch is "develop". For a hotfix, you will use whatever branch name is used for the hotfix. Leave the custom label blank and click "run workflow". This will create an action that should result in a zip file. Inside that zip is another zip that contains the war file. ## Deploy Release Candidate to Internal Dedicated Issue  ssh into the dataverse-internal server and download the release candidate war file you built above. Go to /doc/release-notes, open the release-notes.md file for the release we're working on, and perform all the steps under "Upgrade Instructions". Note that for regular releases, we haven't bumped the version yet so you won't be able to follow the steps exactly. (For hotfix releases, the version will be bumped already.) ## Deploy Release Candidate to QA Dedicated Issue  Deploy the same war file to using the same upgrade instructions as above. ## Solicit Feedback from Curation Team Ask the curation team to test on and give them five days to provide feedback. This is our main form of regression testing for the UI. ## Conduct Performance Testing Dedicated Issue  See {ref}`locust` and , for example. ## Build the Guides for the Release Candidate Go to and make the following adjustments to the config: - Repository URL: `https://github.com/IQSS/dataverse.git` - Branch Specifier (blank for 'any'): `*/develop` - `VERSION` (under "Build Steps"): use the next release version but add "-rc.1" to the end. Don't prepend a "v". Use `6.8-rc.1` (for example) Click "Save" then "Build Now". Make sure the guides directory appears in the expected location such as When previewing the HTML version of docs from pull requests, we don't usually use this Jenkins job, relying instead on automated ReadTheDocs builds. The reason for doing this step now while we wait for feedback from the Curation Team is that it's an excellent time to fix the Jenkins job, if necessary, to accommodate any changes needed to continue to build the docs. For example, Sphinx might need to be updated or a dependency might need to be installed. Such changes should be listed in the release notes for documentation writers. ## Deploy Release Candidate to Demo Dedicated Issue  Time has passed. The curation team has given feedback. Fixes may have been merged into the "develop" branch. We're ready to actually make the release now, which includes deploying a release candidate to the demo server. Build a new war file, if necessary, and deploy it to using the upgrade instructions in the release notes. ## Merge Release Notes (Once Ready) If the upgrade instructions are perfect, simply merge the release notes. If the upgrade instructions aren't quite right, work with the authors of the release notes until they are good enough, and then merge. For a hotfix, there are no release notes to merge yet. ## Prepare Release Branch Dedicated Issue  **Note:** The changes below must be the very last commits merged into the develop branch before it is merged into master and tagged for the release! For a regular release, branch from the "develop" branch and give the branch a name like "12520-bump-to-6.12" like we did at . For a hotfix release, branch from the appropriate tag such as `v6.12` and give the branch a reasonable name. Make the following changes in the release branch. Increment the version number to the milestone (e.g. 6.12) in the following two files: - modules/dataverse-parent/pom.xml -> `` -> `` - doc/sphinx-guides/source/conf.py In the following `versions.rst` file: - doc/sphinx-guides/source/versions.rst - Below the `- |version|` bullet (`|version|` comes from the `conf.py` file you just edited), add a bullet for what is soon to be the previous release. If you are making a regular release, return to the parent pom and make the following change, which is necessary for proper tagging of images: - modules/dataverse-parent/pom.xml -> `` -> profile "ct" -> `` -> Set `` to `${revision}` (Before you make this change the value should be `${parsedVersion.majorVersion}.${parsedVersion.nextMinorVersion}`. Later on, after cutting a release, we'll change it back to that value. See {ref}`base_image_post_release`.) Test the changes in Docker. Note that you will have to build the base image manually. See {ref}`base-image-build-instructions`. If you are making a regular release, you can refer to as an example of the changes described above. Make a pull request. Make sure tests are passing. Have someone approve it. Once we have collectively decided to go forward with the release, merge the pull request. If you are making a hotfix release, `` should already be set to `${revision}`. If so, leave it alone. Go ahead and do the normal bumping of version numbers described above. Make the pull request against the "master" branch. Put it through review and QA, including merging. Do not delete the branch after merging because we will later merge it into the "develop" branch to pick up the hotfix. More on this later. ## Merge "develop" into "master" (non-hotfix only) If this is a regular (non-hotfix) release, create a pull request to merge the "develop" branch into the "master" branch using this "compare" link: After making the pull request, allow time for important tests pass: - Unit tests: Maven Tests - API tests: Container Integration Tests Workflow - JSF tests: Dataverse JSF Frontend Tests Workflow Don't worry about style and quality test failures such as these: - Code scanning results / CodeQL - Maven CheckStyle Task / Checkstyle job - Maven Tests / SonarQube Analysis and Coverage It's ok to skip code review. When merging the pull request, be sure to choose "create a merge commit" and not "squash and merge" or "rebase and merge". We suspect that choosing squash or rebase may have led to [lots of merge conflicts](https://github.com/IQSS/dataverse/pull/11647#issuecomment-3085289132) when we tried to perform this "merge develop to master" step, forcing us to [re-do](https://docs.google.com/document/d/1oit6LLDUWpNpV_uWveOMvdwDsaUey-74ehvzCZp1f3k/edit?usp=sharing) the previous release before we could proceed with the current release. If this is a hotfix release, skip this whole "merge develop to master" step (the "develop" branch is not involved until later). ## Confirm "master" Mergeability Hopefully, the previous step went ok. As a sanity check, use the "compare" link at again to look for merge conflicts without making a pull request. If the GitHub UI tells you there would be merge conflicts, something has gone horribly wrong (again) with the "merge develop to master" step. Stop and ask for help. ## Add Milestone to Pull Requests and Issues Often someone is making sure that the proper milestone (e.g. 6.10.1) is being applied to pull requests and issues, but sometimes this falls between the cracks. Check for merged pull requests that have no milestone by going to and entering [is:pr is:merged no:milestone](https://github.com/IQSS/dataverse/pulls?q=is%3Apr+is%3Amerged+no%3Amilestone) as a query. If you find any, first check if those pull requests are against open pull requests. If so, do nothing. Otherwise, add the milestone to the pull request and any issues it closes. This includes the "merge develop into master" pull request above. (build-guides)= ## Build the Guides for the Release Go to and make the following adjustments to the config: - Repository URL: `https://github.com/IQSS/dataverse.git` - Branch Specifier (blank for 'any'): `*/master` - `VERSION` (under "Build Steps"): bump to the next release. Don't prepend a "v". Use `6.10.1` (for example) Click "Save" then "Build Now". Make sure the guides directory appears in the expected location such as As described below, we'll soon point the "latest" symlink to that new directory. (run-build-create-war)= ## Run a Build to Create the War File Go to click "run workflow". For a regular release, change the branch to "master". For a hotfix release, use whatever branch name is used for the hotfix. Leave the custom label blank and click "run workflow". This will create an action that should result in a zip file. Inside that zip is another zip that contains the war file. Download it. The build number will appear in `/api/info/version` (along with the commit mentioned above) from a running installation (e.g. `{"version":"6.10.1","build":"master-300d5b5"}`). ## Build Installer (dvinstall.zip) In a git checkout of the source, switch to the master branch and pull the latest. Unzip the zip file from the previous step. Copy the war file to the `target` directory in the root of the repo (create the `target` directory, if necessary): ```bash cp ~/Downloads/built-app.zip . unzip built-app.zip rm built-app.zip mkdir -p target mv dataverse-*.war target ``` Then, create the installer: ```bash cd scripts/installer make clean make ``` A zip file called `dvinstall.zip` should be produced. ## Create a Draft Release on GitHub Go to to start creating a draft release. - Under "Select tag" you will be creating a new tag. Have it start with a "v" such as `v6.10.1`. Click "Create new tag". Don't worry, the tag won't be created until you publish. - Under "Target", choose "master". This commit will appear in `/api/info/version` from a running installation. - Under "Release title" use the same name as the tag such as `v6.10.1`. - In the description, copy and paste the content from the release notes .md file created in the "Write Release Notes" steps above. - Under "attach binaries", upload the war file and installer you created above. - Click "Save draft" because we do not want to publish the release yet. At this point you can send around the draft release for any final feedback. Links to the guides for this release should be working now, since you build them above. Make corrections to the draft, if necessary. It will be out of sync with the .md file, but that's ok ([#7988](https://github.com/IQSS/dataverse/issues/7988) is tracking this). ## Publish the Release Click the "Publish release" button. ## Update Guides Link "latest" at is a symlink to the directory with the latest release. That directory (e.g. `6.10.1`) was put into place by the Jenkins "guides" job described above. ssh into the guides server and update the symlink to point to the latest release, as in the example below. ```bash cd /var/www/html/en ln -s 6.10.1 latest ``` This step could be done before publishing the release if you'd like to double check that links in the release notes work. ## Test Docker Images Publishing the release should have trigged the ["Container Images Scheduled Maintenance" GitHub Action](https://github.com/IQSS/dataverse/actions/workflows/container_maintenance.yml). Allow it to finish and then go to and navigate to "gdcc/dataverse". Click on "tags" and look at the "latest" tag. Was it just updated? Good! If not, we plan to address this is but for now, as a workaround, run the action again. Go to and click the "run workflow" dropdown. Make sure the branch is set to "develop" and click "run workflow" button. Wait for the action to finish and then check again that the "latest" tag has been updated. Locally, delete old images and spin up the "latest" tag. ```bash docker rmi gdcc/dataverse:latest docker rmi gdcc/configbaker:latest cd docker/compose/demo rm -rf data docker compose up ``` Wait for the bootstrapping process to complete. Then, look at to make sure "version" shows the version that you just released. Note that it's normal for "build" to be null for our Docker images. ## Close Milestone on GitHub Now that we've published the release, close the [milestone](https://github.com/IQSS/dataverse/milestones). (base_image_post_release)= ## Update the Container Base Image Version Property Dedicated Issue  Create a new branch (any name is fine but `prepare-next-iteration` is suggested) and update the following files to prepare for the next development cycle: - modules/dataverse-parent/pom.xml -> `` -> profile "ct" -> `` -> Set `` to `${parsedVersion.majorVersion}.${parsedVersion.nextMinorVersion}` Create a pull request. Wait for checks to complete. It's ok (even expected) for the following check to fail: - main-integration-tests-workflow from container_integration_tests.yml - If you see an error like `Error: DOCKER> Unable to pull 'gdcc/base:6.12-noble-p7.2026.2-j21' : {"message":"manifest for gdcc/base:6.12-noble-p7.2026.2-j21 not found: manifest unknown: manifest unknown"} (Not Found: 404) [{"message":"manifest for gdcc/base:6.12-noble-p7.2026.2-j21 not found: manifest unknown: manifest unknown"} (Not Found: 404)]` it's telling you that the Docker image can't be spun up for API testing because it doesn't exist yet. (The error above was just after the 6.11 release.) The image will exist once the pull request is approved and merged. Put the pull request through code review, like usual, but make sure reviewers know it's ok to ignore the check above. Give it a milestone of the next release, the one **after** the one we're working on. Once the pull request has been approved, merge it. It should be the first PR merged of the next release. For more background, see {ref}`base-image-supported-tags`. For an example, see For a hotfix, we will do this later and in a different branch. See below. ## Deploy Final Release on Demo Dedicated Issue  Above you already did the hard work of deploying a release candidate to . It should be relatively straightforward to undeploy the release candidate and deploy the final release. (update-schemaspy)= ## Update SchemaSpy We maintain SchemaSpy at URLs like and (for example) Get the attention of the core team and ask someone to update it for the new release. Consider updating [the thread](https://groups.google.com/g/dataverse-community/c/f95DQU-wlVM/m/cvUp3E9OBgAJ) on the mailing list once the update is in place. See also {ref}`schemaspy`. ## Add the Release to the Dataverse Roadmap If shows a list of releases, add it to. ## Announce the Release Dedicated Issue  - Make a blog post at - Post a message at - Post a message under #community at ## For Hotfixes, Merge Hotfix Branch into "develop" Note: this only applies to hotfixes! We've merged the hotfix into the "master" branch but now we need the fixes (and version bump) in the "develop" branch. Make a new branch off the hotfix branch. You can call it something like "6.7.1-merge-hotfix-to-develop". In that branch, do the {ref}`base_image_post_release` step you skipped above. Now is the time. Create a pull request against develop. Merge conflicts are possible and this pull request should go through review and QA like normal. Afterwards it's fine to delete this branch and the hotfix branch that was merged into master. ## For Hotfixes, Tell Developers to Merge "develop" into Their Branches and Rename SQL Scripts Note: this only applies to hotfixes! Because we have merged a version bump from the hotfix into the "develop" branch, any SQL scripts in the "develop" branch should be renamed (from "5.11.0" to "5.11.1" for example). (To read more about our naming conventions for SQL scripts, see {doc}`sql-upgrade-scripts`.) Look at `src/main/resources/db/migration` in the "develop" branch and if any SQL scripts have the wrong version, make a pull request (or ask a developer to) to update them (all at once in a single PR is fine). Tell developers to merge the "develop" into their open pull requests (to pick up the new version and any fixes) and rename SQL scripts (if any) with the new version. ## Alert Translators About the New Release Create an issue at to say a new release is out and that we would love for the properties files for English to be added. For example, for 6.11 we wrote "Update en_US/Bundle.properties etc. for Dataverse 6.11" at ## Lift the Code Freeze and Encourage Developers to Update Their Branches First, double check that the pull request that contains the change in the {ref}`base_image_post_release` step has been merged. It's now safe to lift the code freeze. We can start merging pull requests into the "develop" branch for the next release. Let developers know that they should merge the latest from the "develop" branch into any branches they are working on. (For hotfixes we've already told them this.)