Using Test Data Packages

With test data packages, you can migrate data between two Vaults. Test data packages are particularly useful when your organization needs to test configuration in a pre-release Vault before an upcoming general release or preserve object data when refreshing a sandbox Vault. Test data packages leverage the Global ID and Link fields to migrate data.

If you only need to migrate configuration changes, use a migration package. If you need to migrate large amounts of object data or documents between Vaults, we recommend using Vault Loader.

How to Enable Test Data Packages

You can enable this feature from Admin > Settings > General Settings. There are two settings involved:

  • Allow Outbound Packages: This setting must be on for the source Vault. This setting makes the Admin > Deployment tab visible.
  • Allow Inbound Packages: This setting must be on for the target Vault. This setting makes the Admin > Deployment tab visible.

After enabling, you may need to log out and log back in for the Deployment tab to appear.

How to Create & Export Test Data Packages

To create an outbound test data package:

  1. Navigate to Admin > Deployment > Outbound Packages and click Create.
  2. In the dialog, select Test Data as the Outbound Package Type.
  3. Enter a Summary to help you and other users understand the purpose and contents of this package.
  4. Optional: Select a user as Owner. If you leave this blank, it defaults to you. When importing the package, owner information appears in the target Vault to provide a contact person if there are questions about the package.
  5. Optional: Provide the User Name of a Test Data User. Because user data is unlikely to be the same in different Vaults, this user will be used to map User references in the target Vault.
  6. Click Save. Vault opens the package details page.
  7. Navigate to the Data section. Click Add to add a new dataset to the outbound package. Enter a Set Label. If you leave this field blank, Vault will automatically generate a label based on the selected object. Select a data object from the Primary Object picklist. To add a related data object, click Add Related Object and select a related object. You can add up to nine (9) related objects to each dataset. Click Save to return to the Outbound Packages page.
  8. Optional: Navigate to the Packages section. Click Upload to store the exported VPK.
  9. In the Actions menu, select Export. While exporting, Vault automatically generates a value for the Link field, overwriting the existing value. When the export is complete, a notice appears on your Notifications page and Vault sends you an email. The notifications include a link to download your VPK. Vault does not automatically download the VPK to your machine.

About Datasets

You can use datasets within an outbound test data package to migrate object data. Datasets consist of object and related object data. Each dataset can contain up to ten (10) data objects, including one (1) primary object and up to nine (9) additional related objects. Each outbound package can contain up to 100 objects.

Datasets can also include object relationships. For example, the grandparent, parent, and child relationship between Study, Country, and Site.

Although the User object is available in test data packages, user data is unlikely to be the same in different Vaults, especially Vaults in different domains. Therefore, we recommend using Vault Loader to extract and move user records to another Vault in most cases.

Selecting Objects

To add objects to a dataset, from the Dataset Configuration page:

  1. Select a Primary Object from a list of all available objects in your Vault. This list excludes object types and objects with system-managed records. If Groups is selected as the Primary Object, review the considerations for adding groups to a dataset.
  2. Optional: Click Add Related Objects to select from a list of applicable outbound and inbound relationships.
  3. Click Save.

Editing Datasets

To edit an existing dataset, select Edit from the Actions menu next to the dataset on the Outbound Package detail page, or click Edit on the Dataset Configuration page.

Editing Default Settings for Data Objects

To edit the default settings of data objects:

  1. From the Outbound Package detail page, click the dataset.
  2. From the desired object, select Edit from the Actions menu.
  3. Optional: Select Upsert, Create, Delete, or Update from the Action picklist. This determines the default action for deployment. Vault sets Upsert as the default action.
  4. Optional: Select a field from the Key Field picklist. This is a unique field used for looking up a record for the update portion of the Upsert, Update, or Delete actions. The Create action does not require a unique field for lookup.
  5. Optional: Select the Record Migration Mode checkbox. Record Migration Mode allows you to migrate object records in a noninitial state for objects with lifecycles. Additionally, Vault bypasses entry criteria, entry actions, validation rules, event actions, and reference constraints when creating or updating records in migration mode. Additionally, you can use Record Migration Mode to create inactive records and references to inactive records.
  6. Optional: Expand the Filter section to apply filters by entering a VQL WHERE clause. This field is used to filter down the records you would like to move, using the same syntax as the Vault Loader Where Clause field. If the object has an inbound child relationship to another object, Vault automatically includes a status__v='active' filter, which you should not remove. Click Validate to check for syntax errors. Learn more about VQL in the Developer Portal.
  7. Optional: Expand the Columns section to modify columns for the selected object. All editable columns are selected by default. You can choose not to move certain field values over by deselecting the columns to include. You can choose to replace the default reference field for an outbound relationship with any other unique reference field, however, you can only include one reference field for each outbound relationship. If there are no valid unique reference fields available, Vault excludes those columns. If the object has an inbound child relationship to another object, Vault excludes the status__v column.
  8. Click Save.

Adding Groups to Datasets

If groups are included in a dataset, consider the following limitations:

  • You cannot add related objects to a dataset with Groups as the Primary Object.
  • You cannot create system-managed groups, but you can update existing system-managed groups.
  • You cannot export, create, or update Auto Managed Groups.
  • You cannot export group members. However, Vault may add or remove group members automatically depending on the security profile included for the group.

How to Import & Validate Test Data Packages

To import and validate a test data package:

  1. Navigate to Admin > Deployment > Inbound Packages and click Import.
  2. Find and select the exported VPK file on your computer.
  3. Vault imports and validates the package asynchronously and sends you a notification by email when complete. Click the link in the notification email to open the validation log for the import.
  4. To validate an existing inbound package, select Validate from the Actions menu next to the inbound package Name.

About the Validation Log

The validation log contains details about the deployment status and dependencies for each package step in an inbound package.

How to Deploy a Test Data Package

  1. From the Actions menu of the inbound package, select Review & Deploy. Vault displays a list of all components in the package.
  2. Review the Deployment Status and Deployment Action for each item. To exclude specific steps, clear their checkbox.
  3. Optional: To reorder steps, click Reorder, enter the desired step number in the Step field, then click Save.
  4. Click Next.
  5. On the confirmation page, review and click Finish.

Deployment Status

Vault determines the Deployment Status of an inbound package first by verifying that each step (component or data) in the package matches (via checksum) the step in the source Vault. The status displays when you view an inbound package and on the Review and Select Steps and Confirmation steps during Review & Deploy. Once you deploy, the status updates to reflect what happened. Deployment Status exists on each step and on the package as a whole. If an individual step encounters an error, the package’s status shows Error as well.

  • Imported: The VPK was successfully imported.
  • Verified: The package’s steps match the source Vault, but have not yet been deployed.
  • Not Verified: Indicates that the VPK or step was altered.
  • Deployed: Vault successfully deployed the component’s configuration on the target Vault.
  • In Deployment: VPK deployment is in progress.
  • Deployed: The VPK was successfully deployed without warnings or errors.
  • Deployed with Failures: Vault fully deployed VPK with some failures.
  • Deployed with Errors: Vault identified errors during deployment, but was still able to successfully deploy some components and data.
  • Skipped: When deploying, the user chose to skip this step or Vault skipped this step because the component already existed in the target Vault.
  • Error: Vault encountered an error when deploying the package. See error details.

Deployment Action

Vault determines the Deployment Action of an inbound package based on the inbound package steps. Deployment Action displays on the Review and Select Steps and Deployment Confirmation steps during Review & Deploy. Unlike Deployment Status, Deployment Action only exists at the step level, for each step of the package.

  • Create: This is a new data step that the package will add to the target Vault.
  • Upsert: This operation adds a data step to the target Vault if it does not already exist, or updates if it does.
  • Update: This component already exists on the target Vault, but its configuration is not identical. The package will update the existing component to match. This operation maintains all references to the component, such as the audit trail.

Deployment Errors

Although most deployment errors can be avoided by reviewing the Validation Log, deployment errors can still occur. When Vault encounters an error during a test data package deployment, it stops the deployment but does not revert any data steps it has already deployed.

To resolve the error and resume the deployment:

  1. Review the deployment log to understand what caused the error. A link to this appears in the notification. The log is also attached to the inbound package. You can download it from the inbound package’s detail page.
  2. Within the deployment log, copy and paste the link from a specific deployment step into your browser to download the success and failure logs. To download the success and failure logs of all deployment steps, copy and paste the link from the deployment summary into your browser. These failure logs are the same as those returned by Vault Loader.
  3. Return to the package with the error in Inbound Packages. From the actions menu, choose Deploy.
  4. Vault restarts the process where it stopped on the previous deployment.

Test Data Package Limitations

Consider the following limitations when using test data packages:

  • A maximum of 10 related objects can be included in a single dataset.
  • A maximum of 100 objects can be included in a single test data package.
  • Test data packages cannot support the extraction and upload of documents.
  • Document references on object records are excluded.
  • Object records with self-referencing relationships require you to use more than one dataset. Otherwise, these steps will fail during deployment.
  • Required object fields in the target Vault that do not exist in the source Vault should be made non-required. Otherwise, these steps will fail during deployment.

The following permissions control actions for test data packages:

Type Permission Label Controls
Security Profile Admin: Migration Packages: Create Ability to create packages for configuration or data migration; you'll need this permission on the source Vault.
Security Profile Admin: Migration Packages: Deploy Ability to deploy packages for configuration or data migration; you'll need this permission on the target Vault.
Security Profile Admin: Various Ability to create or edit various components. When deploying packages, you'll need the same permissions that you'd need if you were manually creating the components, for example, Admin > Picklists > Edit to deploy a package that includes the picklist component.
Security Profile Objects: Outbound Package: Read Ability to view the Outbound Package object. You must also have Read permission on all fields for this object.
Security Profile Objects: Inbound Package: Read Ability to view the Inbound Package object. You must also have Read permission on all fields for this object.
Security Profile Objects: Inbound Package Step: Read Ability to view the Inbound Package Step object. You must also have Read permission on all fields for this object.
Security Profile Objects: Inbound Package Data: Read Ability to view the Inbound Package Data object. You must also have Read permission on all fields for this object.
Security Profile Objects: Inbound Package Component: Read Ability to view the Inbound Package Component object. You must also have Read permission on all fields for this object.

The standard Vault Owner and System Admin security profiles have all of the necessary permissions.