Automate XM Cloud Maintenance Tasks

When working with XM Cloud, there is a fundamental difference in how serialized items are handled during a deployment. This can lead to some unwanted side effects with your data or templates. In this blog post, we explore how you can automate some maintenance tasks to avoid unexpected situations.

Deploying serialized items in XM Cloud

In the past, you might have automated the process of pushing the serialized items into a running Sitecore instance during a deployment. This synchronization process would add Sitecore items to the database.

With XM Cloud, your deployments are handled by XM Cloud Deploy. The build step of a deployment process takes the serialized items and puts them in resource files. At runtime, items from resource files and databases are merged into the content tree. The items themselves are never actually added to the database, until you modify them in the CMS.

Unwanted side effects

As soon as you change and save a serialized item, the entire item is copied to the database and it will have precedence over changes coming from resource files. The consequence being that the item will not be updated anymore in that specific environment even when the underlying serialized item changes during a deployment.

Especially in a development environment, where you might frequently change items for testing or development purposes, this creates a rather unpredictable state. Luckily, Sitecore provides you all the tools you need to handle this.

The Items as Resource Plugin

The Sitecore CLI Items as Resource plugin includes an itemres command. For our example, we will focus on the cleanup subcommand.

Initially, the cleanup subcommand would only clean up database items in resource files if the item data in both entries were equal. The --force option introduced in version 5.1.28 will do an item cleanup without comparing them. We can leverage this option to reset serialized items to their original state.

Automate the Cleanup Steps

This example will use Azure DevOps pipelines and the Sitecore CLI. As the heavy lifting is done by the Sitecore CLI, you can build this in any other tool as well.

The pipelines resets individual items, as well as certain item paths. The following template shows the steps needed to authenticate with XM Cloud, connect to a specific environment and then loop over paths to reset either indiviually or recursively.

parameters:
  - name: environmentName
    type: string
  - name: environmentId
    type: string
  - name: cleanupPaths
    type: object
    default: []
  - name: cleanupPathsRecursive
    type: object
    default: ['/sitecore/templates']

steps:
  - script: "env | sort"
    displayName: "Display environment variables"

  - script: "dotnet tool restore"
    displayName: "Restoring Sitecore CLI"

  - script: "dotnet sitecore --help"
    displayName: "Installing Sitecore CLI Plugins"

  - script: "dotnet sitecore --version"
    displayName: "Show Sitecore CLI Version"

  - script: "dotnet sitecore cloud login --client-credentials --client-id $(XM_CLOUD_CLIENT_ID) --client-secret $(XM_CLOUD_CLIENT_SECRET) --allow-write"
    displayName: "Authenticate CLI with XM Cloud"

  - script: "dotnet sitecore cloud environment connect -id ${{ parameters.environmentId }} --allow-write"
    displayName: "Connect the CLI to the Environment"

  - ${{ each cleanupPath in parameters.cleanupPaths }}:
    - script: 'dotnet sitecore itemres cleanup -n "${{ parameters.environmentName }}" -p "${{ cleanupPath }}" --force -v -t'
      displayName: "Cleanup path ${{ cleanupPath }}"

  - ${{ each cleanupPath in parameters.cleanupPathsRecursive }}:
    - script: 'dotnet sitecore itemres cleanup -n "${{ parameters.environmentName }}" -p "${{ cleanupPath }}" --force -r -v -t'
      displayName: "Cleanup path (recursive) ${{ cleanupPath }}"

The Maintenance Pipeline

In our setup, we run the maintenance pipeline automatically every night on a default set of paths. We also include the option to publish after the cleanup steps. The pipeline supports parameters, so you can run the pipeline manually at any time for additional or different paths.

Similar to the previous blog post about how to sync content between XM Cloud environments automatically, we run this task in parallel on multiple environments using stages.

parameters:
  - name: cleanupXmCloud
    displayName: Run XM Cloud Cleanup?
    type: boolean
    default: true
  - name: cleanupPaths
    displayName: Paths to cleanup
    type: object
    default:
      - /sitecore/content/MySiteCollection
  - name: cleanupPathsRecursive
    displayName: Paths to cleanup (recursive)
    type: object
    default:
      - /sitecore/templates
      - /sitecore/system
      - /sitecore/layout
      - /sitecore/content/MySiteCollection/MySite/Presentation      
      - /sitecore/content/MySiteCollection/MySiteInventory
  - name: publishXmCloud
    displayName: Publish to Experience Edge?
    type: boolean
    default: true
  - name: fullPublish
    displayName: Run full publish?
    type: boolean
    default: true
  - name: publishPaths
    displayName: Paths to publish
    type: object
    default:
      - /sitecore/templates
      - /sitecore/system
      - /sitecore/layout
      - /sitecore/content/MySiteCollection/MySite/Dictionary
      - /sitecore/content/MySiteCollection/MySite/Presentation
      - /sitecore/content/MySiteCollection/MySite/Settings
      - /sitecore/content/MySiteCollection/MySiteInventory
  - name: publishSubitems
    displayName: Include Subitems?
    type: boolean
    default: true
  - name: skipProduction
    displayName: Skip all Tasks on Production Environment?
    type: boolean
    default: false

trigger: none

schedules:
- cron: '0 1 * * *'
  displayName: Continuous Maintenance (Nightly at 1 AM)
  branches:
    include:
      - develop
  always: true

variables:
  - template: /azure/azure-templates/variables.yml
  - group: xmcloud-global
  - name: environment
    value: $(XM_CLOUD_ENVIRONMENT_NAME)
  - name: environmentId
    value: $(XM_CLOUD_ENVIRONMENT_ID)

stages:
  - stage: Maintenance_Development
    displayName: Run Maintenance on Development
    variables:
      - group: xmcloud-development
    jobs:
      - job: Maintenance_Cleanup
        displayName: Cleanup Data
        condition: eq(${{ parameters.cleanupXmCloud }}, 'true')
        steps:
          - template: /azure/azure-templates/steps-cleanup-xmcloud.yml
            parameters:
              environmentName: $(environment)
              environmentId: $(environmentId)
              cleanupPaths: ${{ parameters.cleanupPaths }}
              cleanupPathsRecursive: ${{ parameters.cleanupPathsRecursive }}

      - job: Maintenance_Publish
        displayName: Publishing Data
        dependsOn:
          - Maintenance_Cleanup
        condition: |
          and
          ( 
            eq(${{ parameters.publishXmCloud }}, 'true'),        
            in(dependencies.Maintenance_Cleanup.result, 'Succeeded', 'SucceededWithIssues', 'Skipped')
          )
        steps:
          - template: /azure/azure-templates/steps-publish-xmcloud.yml
            parameters:
              environmentName: $(environment)
              environmentId: $(environmentId)
              fullPublish: ${{ parameters.fullPublish }}
              publishPaths: ${{ parameters.publishPaths }}
              publishSubitems: ${{ parameters.publishSubitems }}

  - stage: Maintenance_Staging
    displayName: Run Maintenance on Staging
    variables:
      - group: xmcloud-staging
    dependsOn: []
    jobs:
      - job: Maintenance_Cleanup
        displayName: Cleanup Data
        condition: eq(${{ parameters.cleanupXmCloud }}, 'true')
        steps:
          - template: /azure/azure-templates/steps-cleanup-xmcloud.yml
            parameters:
              environmentName: $(environment)
              environmentId: $(environmentId)
              cleanupPaths: ${{ parameters.cleanupPaths }}
              cleanupPathsRecursive: ${{ parameters.cleanupPathsRecursive }}

      - job: Maintenance_Publish
        displayName: Publishing Data
        dependsOn:
          - Maintenance_Cleanup
        condition: |
          and
          ( 
            eq(${{ parameters.publishXmCloud }}, 'true'),        
            in(dependencies.Maintenance_Cleanup.result, 'Succeeded', 'SucceededWithIssues', 'Skipped')
          )
        steps:
          - template: /azure/azure-templates/steps-publish-xmcloud.yml
            parameters:
              environmentName: $(environment)
              environmentId: $(environmentId)
              fullPublish: ${{ parameters.fullPublish }}
              publishPaths: ${{ parameters.publishPaths }}
              publishSubitems: ${{ parameters.publishSubitems }}

  - stage: Maintenance_Production
    displayName: Run Maintenance on Production
    condition: ne(${{ parameters.skipProduction }}, 'true')
    variables:
      - group: xmcloud-production
    dependsOn: []
    jobs:
      - job: Maintenance_Cleanup
        displayName: Cleanup Data
        condition: eq(${{ parameters.cleanupXmCloud }}, 'true')
        steps:
          - template: /azure/azure-templates/steps-cleanup-xmcloud.yml
            parameters:
              environmentName: $(environment)
              environmentId: $(environmentId)
              cleanupPaths: ${{ parameters.cleanupPaths }}
              cleanupPathsRecursive: ${{ parameters.cleanupPathsRecursive }}

      - job: Maintenance_Publish
        displayName: Publishing Data
        condition: eq(${{ parameters.publishXmCloud }}, 'true')
        dependsOn:
          - Maintenance_Cleanup
        steps:
          - template: /azure/azure-templates/steps-publish-xmcloud.yml
            parameters:
              environmentName: $(environment)
              environmentId: $(environmentId)
              fullPublish: ${{ parameters.fullPublish }}
              publishPaths: ${{ parameters.publishPaths }}
              publishSubitems: ${{ parameters.publishSubitems }}

Conclusion

Combining these maintenance tasks with syncing content automatically between XM Cloud environments helped us keeping environments in a predictable and up-to-date state with production content. At the same time you can use an environment to test changes, even adjust a template or a rendering directly in the XM Cloud instance without having to worry too much about side effects.