Deep Dive Into Negative Change Sets

"Shannon, I don't know what you just released to prod, but our CEO is calling me directly. Please fix it."

...

Ever wish you could just click a single button and undo a deployment? Well, you're in luck.

One of the topics I covered at Dreamforce with Norman Krishna during our "Where's the Undo Button" session was Negative Change Sets. 

Negative Change Sets are a declarative way to completely undo a release with a single button click. Why would you want to do this?

Let's say you deploy great items into production, but they somehow manage to negatively impact your users. Your immediate options are:

1. Try to fix whatever you broke on the fly
2. Revert back to a prior version of the functionality you created and remove the new functionality/items you deployed

As you can imagine, option #1 is super risky with no guarantee you'll be able to fix items. You'll also be in trouble with compliance if you're changing things directly in production, so you'll have to repeat the issue in a sandbox, fix it in the sandbox, and deploy your fix into production, which can take extra time.

With option #2, there is significantly less risk, but it may still take a while to revert because change sets take time to build and validate - especially if you have Apex in your org. You also need to make sure that anything touched in your deployment is reverted, which can be a time consuming and tedious task under pressure.

Enter Negative Change Sets. 

Instead of reverting in the moment, let's *plan* for our deployment to fail, and build our change sets before our deployment ever occurs. 

When you build and deploy a negative change set, you are reverting production back to the state it was in before your release. Ideally, your users will know no difference.

The idea is to build one or two change sets of the items that will undo your release, then quick deploy them if necessary. This means you won't have to scurry to fix things, then wait for the validation to occur. Planning to simply undo the release completely provides you more time to figure out what may have went wrong, and properly test any additional modifications in a sandbox.

Here are detailed instructions to build and deploy a negative change set. 

1. Figure out what you intend to deploy to production in your next release. 
• For our example, we intend to release a contact process builder change and an account page layout change, a new patient record object with several fields, and a new validation rule on opportunities
2. Refresh a brand new, unused developer sandbox from production, call it "Negative Box" or something to denote this is where you will be building negative change sets
3. Allow the new Negative Box sandbox to deploy changes to Production
4. Create a brand new change set in Negative Box. Call it "Negative Box Revert" and add a release number or any other crucial details to the description to ensure everyone will know what this change set is -- a one-click undo button for the release.
5. Add all Apex, Visualforce, Lightning Components, etc. to the Negative Box Revert change set
• This ensures that if we make code changes during our next release, the code will be reverted back to a stable state, where we will achieve code coverage
• If you have made code changes, check our Caveats section below.
6. Add all of the existing items wherein you have changes or updates to the Negative Box Revert change set, which may include:
• existing process builders
• existing flows
• existing workflow rules
• existing fields
• existing validation rules
• existing approval processes
• existing profiles
7. For brand new functionality or objects, push them to Negative Box so you can use a change set from Negative Box to hide them with a single click
• For our example, the brand new functionality and objects are the patient record object and our opportunity validation rule
• We will bundle the patient record object and the opportunity validation rule into a change set, then deploy into Negative Box
8. In Negative Box, remove brand new items from page layouts, modify the field level security to hide them, and remove permissions from profiles to revert - just like you would if you needed to hide/revert in production
• For our example, we are hiding the patient record object from all profiles except System Administrator, and we are deactivating the new opportunity validation rule
9. Bundle all new items into the Negative Box Revert change set. Include profiles, since you have likely hidden objects at the profile level as well.
1. In our example, we include the opportunity validation rule and the patient record object, as well as all of our profiles
10. Push the Negative Box Revert change set into production and validate. 
11. Deploy the Negative Box Revert change set only as needed to revert.

Caveats:

• Process Builders and Flows do not always automatically activate when updating or reverting an existing version. Ensure you check all versions to maintain the preferred active version.
• For code changes, you should use the current Negative Box code versions to completely revert to existing production state. Then, bundle into the Negative Box Revert change set.
• For brand new code, move the code into Negative Box but comment it out. Double check test classes to ensure they do not fail. Deactivate brand new triggers in Negative Box. Then, bundle into the Negative Box Revert change set.
• If you have hidden new items or functionality during your Negative Change Set, you should return to the items or functionality in a patch release, OR completely delete them at a later date -- there is no need to keep broken, hidden items or functionality in your production instance

In conclusion, a little planning goes a long way. Let Negative Change Sets save your next deployment - before anything blows up. 

P.S. There are even cooler tricks to talk about regarding reversion. This post is meant for admins who are familiar with declarative functionality, but have never heard of source control. More to come, for sure!