Publishing and non-publishing repositories¶

The key to shared mutable history is to keep your changesets in draft phase as you pass them around. Recall that by default, hg push promotes changesets from draft to public, and public changesets are immutable. You can change this behaviour by reconfiguring the remote repository so that it is non-publishing. (Short version: set phases.publish to false. Long version follows.)

Setting up¶

We’ll work through an example with three local repositories, although in the real world they’d most likely be on three different computers. First, the public repository is where tested, polished changesets live, and it is where you synchronize with the rest of your team.

$ hg init public

We’ll need two clones where work gets done, test-repo and dev-repo:

$ hg clone public test-repo
updating to branch default
0 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ hg clone test-repo dev-repo
updating to branch default
0 files updated, 0 files merged, 0 files removed, 0 files unresolved

dev-repo is your local machine, with GUI merge tools and IDEs and everything configured just the way you like it. test-repo is the test server in a rack somewhere behind SSH. So for the most part, we’ll develop in dev-repo, push to test-repo, test and polish there, and push to public.

The key to shared mutable history is to make the target repository, in this case test-repo, non-publishing. And, of course, we have to enable the evolve extension in both test-repo and dev-repo.

First, edit the configuration for test-repo:

$ hg -R test-repo config --edit --local

and add

[phases]
publish = false

[extensions]
evolve =

Then edit the configuration for dev-repo:

$ hg -R dev-repo config --edit --local

and add

[extensions]
evolve =

Keep in mind that in real life, these repositories would probably be on separate computers, so you’d have to login to each one to configure each repository.

To start things off, let’s make one public, immutable changeset:

$ cd test-repo
$ echo 'my new project' > file1
$ hg add file1
$ hg commit -m 'create new project'
$ hg push
[...]
added 1 changesets with 1 changes to 1 files

and pull that into the development repository:

$ cd ../dev-repo
$ hg pull -u
[...]
added 1 changesets with 1 changes to 1 files
1 files updated, 0 files merged, 0 files removed, 0 files unresolved

Example 1: Amend a shared changeset¶

Everything you learned in the user guide applies to work done in dev-repo. You can commit, amend, uncommit, evolve, and so forth just as before.

Things get different when you push changesets to test-repo. Or rather, things stay the same, which is different: because we configured test-repo to be non-publishing, draft changesets stay draft when we push them to test-repo. Importantly, they’re also draft (mutable) in test-repo.

Let’s commit a preliminary change and push it to test-repo for testing.

$ echo 'fix fix fix' > file1
$ hg commit -m 'prelim change'
$ hg push ../test-repo

At this point, dev-repo and test-repo have the same changesets in the same phases:

Figure 1 a: dev-repo has rev 0:0dc9 public, rev 1:f649 draft

Figure 1 b: the same exact picture in test-repo

(You may notice a change in notation from the user guide: now changesets are labelled with their revision number and the first four digits of the 40-digit hexadecimal changeset ID. Mercurial revision numbers are not consistent when working across repositories, especially when obsolescence is involved. We’ll see why shortly.)

Now let’s switch to test-repo to test our change:

$ cd ../test-repo
$ hg update

Don’t forget to hg update! Pushing only adds changesets to a remote repository; it does not update the working directory (unless you have a hook that updates for you).

Now let’s imagine the tests failed because we didn’t use proper punctuation and capitalization (oops). Let’s amend our preliminary fix (and fix the lame commit message while we’re at it):

$ echo 'Fix fix fix' > file1
$ hg amend -m 'fix bug 37'

For the sake of illustrating how obsolete changesets are not exchanged, let’s amend again:

$ echo 'Fix fix fix.' > file1
$ hg amend -m 'fix bug 37'

Note: some time ago, evolve used to create temporary amend commits. Here, amending twice in a row is reminiscent of that behaviour that you may have seen in older tutorials.

Now we’re in a funny intermediate state (figure 2): revision 1:f649 is obsolete in test-repo, having been replaced by revision 2:96d8 and then by 3:522d—but dev-repo knows nothing of these recent developments.

Figure 2 a: dev-repo same as in Figure 01 a

Figure 2 b: test-repo has rev 0:0dc9 public, revs 1:f649 and 2:96d8 obsolete, rev 3:522d draft

Let’s resynchronize:

$ cd ../dev-repo
$ hg pull -u
[...]
added 1 changesets with 1 changes to 1 files (+1 heads)
2 new obsolescence markers
obsoleted 1 changesets
new changesets 522d503432a2 (1 drafts)
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
updated to "522d503432a2: fix bug 37"
1 other heads for branch "default"

As seen in figure 3, this transfers the new changeset and the obsolescence marker for revision 1. However, it does not transfer 2:96d8, because it is hidden. Push and pull transfer obsolescence markers between repositories, but they do not transfer hidden changesets.

Figure 3: dev-repo receives rev 2:522d, 1:f649 is now obsolete like in test-repo

Because of this deliberately incomplete synchronization, revision numbers in test-repo and dev-repo are no longer consistent. We must use changeset IDs.

Example 2: Amend again, locally¶

This process can repeat. Perhaps you figure out a more elegant fix to the bug, and want to mutate history so nobody ever knows you had a less-than-perfect idea. We’ll implement it locally in dev-repo and push to test-repo:

$ echo 'Fix, fix, and fix' > file1
$ hg amend
$ echo 'Fix, fix, and fix.' > file1
$ hg amend
$ hg push

This time around, the first amend commit stays in dev-repo, and it is not transferred to test-repo—the same as before, just in the opposite direction. Figure 4 shows the two repositories after amending in dev-repo and pushing to test-repo.

Figure 4 a: in dev-repo the extra amend commit is 3:e42b

Figure 4 b: and in test-repo it is 2:96d8

Let’s hop over to test-repo to test the more elegant fix:

$ cd ../test-repo
$ hg update
1 files updated, 0 files merged, 0 files removed, 0 files unresolved

This time, all the tests pass, so no further amending is required. This bug fix is finished, so we push it to the public repository:

$ hg push
[...]
added 1 changesets with 1 changes to 1 files
4 new obsolescence markers

Note that only one changeset—the final version, after all the amendments—was actually pushed. Again, Mercurial doesn’t transfer hidden changesets on push and pull.

So the picture in public is much simpler than in either dev-repo or test-repo. Neither of our missteps nor our amendments are publicly visible, just the final, beautifully polished changeset:

Figure 5: in public repo with revs 0:0dc9 and 1:7b49, both public

There is one important step left to do. Because we pushed from test-repo to public, the pushed changeset is in public phase in those two repositories. But dev-repo has been out-of-the-loop; changeset 7b49 is still draft there. If we’re not careful, we might mutate history in dev-repo, obsoleting a changeset that is already public. Let’s avoid that situation for now by pushing up to dev-repo:

$ hg push ../dev-repo
pushing to ../dev-repo
searching for changes
no changes found

Even though no changesets were pushed, Mercurial still pushed obsolescence markers and phase changes to dev-repo.

A final note: since this fix is now public, it is immutable. It’s no longer possible to amend it:

$ hg amend -m 'fix bug 37'
abort: cannot amend public changesets

This is, after all, the whole point of Mercurial’s phases: to prevent rewriting history that has already been published.

Setting up¶

To demonstrate, let’s start with the public repository as we left it in the last example, with two immutable changesets (figure 5 above). We’ll clone a review repository from it, and then Alice and Bob will both clone from review.

$ hg clone public review
updating to branch default
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ hg clone review alice
updating to branch default
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ hg clone review bob
updating to branch default
1 files updated, 0 files merged, 0 files removed, 0 files unresolved

We need to configure Alice’s and Bob’s working repositories to enable evolve. First, edit Alice’s configuration with

$ hg -R alice config --edit --local

and add

[extensions]
evolve =

Then edit Bob’s repository configuration:

$ hg -R bob config --edit --local

and add the same text.

Example 3: Alice commits and amends a draft fix¶

We’ll follow Alice working on a bug fix. We’re going to use bookmarks to make it easier to understand multiple branch heads in the review repository, so Alice starts off by creating a bookmark and committing her first attempt at a fix:

$ hg bookmark bug15
$ echo 'fix' > file2
$ hg commit -A -u alice -m 'fix bug 15 (v1)'
adding file2

Note the unorthodox “(v1)” in the commit message. We’re just using that to make this tutorial easier to follow; it’s not something we’d recommend in real life.

Of course Alice wouldn’t commit unless her fix worked to her satisfaction, so it must be time to solicit a code review. She does this by pushing to the review repository:

$ hg push -B bug15
[...]
added 1 changesets with 1 changes to 1 files
exporting bookmark bug15

(The use of -B is important to ensure that we only push the bookmarked head, and that the bookmark itself is pushed. See this guide to bookmarks, especially the Sharing Bookmarks section, if you’re not familiar with bookmarks.)

Some time passes, and Alice receives her code review. As a result, Alice revises her fix and submits it for a second review:

$ echo 'Fix.' > file2
$ hg amend -m 'fix bug 15 (v2)'
$ hg push
[...]
added 1 changesets with 1 changes to 1 files (+1 heads)
1 new obsolescence markers
obsoleted 1 changesets
updating bookmark bug15

Figure 6 shows the state of the review repository at this point.

Figure 6: rev 2:4e96 is Alice’s obsolete v1, rev 3:3363 is her v2; both children of rev 1:7b49

After a busy morning of bug fixing, Alice stops for lunch. Let’s see what Bob has been up to.

Example 4: Bob implements and publishes a new feature¶

Meanwhile, Bob has been working on a new feature. Like Alice, he’ll use a bookmark to track his work, and he’ll push that bookmark to the review repository, so that reviewers know which changesets to review.

$ cd ../bob
$ echo 'stuff' > file1
$ hg bookmark featureX
$ hg commit -u bob -m 'implement feature X (v1)'          # rev 4:c7ff
$ hg push -B featureX
[...]
added 1 changesets with 1 changes to 1 files (+1 heads)
exporting bookmark featureX

When Bob receives his code review, he improves his implementation a bit, amends, and submits the resulting changeset for review:

$ echo 'do stuff' > file1
$ hg amend -m 'implement feature X (v2)'                  # rev 5:1bb4
$ hg push
[...]
added 1 changesets with 1 changes to 1 files (+1 heads)
1 new obsolescence markers
obsoleted 1 changesets
updating bookmark featureX

Unfortunately, that still doesn’t pass muster. Bob’s reviewer insists on proper capitalization and punctuation.

$ echo 'Do stuff.' > file1
$ hg amend -m 'implement feature X (v3)'                  # rev 6:9d21

On the bright side, the second review said, “Go ahead and publish once you fix that.” So Bob immediately publishes his third attempt:

$ hg push ../public
[...]
added 1 changesets with 1 changes to 1 files
2 new obsolescence markers

It’s not enough just to update public, though! Other people also use the review repository, and right now it doesn’t have Bob’s latest amendment (“v3”, revision 6:9d21), nor does it know that the predecessor of that changeset (“v2”, revision 5:1bb4) is obsolete. Thus, Bob pushes to review as well:

$ hg push ../review
[...]
added 1 changesets with 1 changes to 1 files (+1 heads)
1 new obsolescence markers
obsoleted 1 changesets
updating bookmark featureX

Figure 7 shows the result of Bob’s work in both review and public.

Figure 7 a: review includes Alice’s draft work on bug 15, as well as Bob’s v1, v2, and v3 changes for feature X: v1 and v2 obsolete, v3 public

Figure 7 b: public contains only the final, public implementation of feature X

Incidentally, it’s important that Bob push to public before review. If he pushed to review first, then revision 6:540b would still be in draft phase in review, but it would be public in both Bob’s local repository and the public repository. That could lead to confusion at some point, which is easily avoided by pushing first to public.

Example 5: Alice integrates and publishes¶

Finally, Alice gets back from lunch and sees that the carrier pigeon with her second review has arrived (or maybe it’s in her email inbox). Alice’s reviewer approved her amended changeset, so she pushes it to public:

$ hg push ../public
[...]
remote has heads on branch 'default' that are not known locally: 9d21d673314a
abort: push creates new remote head 3363442626b3 with bookmark 'bug15'!
(pull and merge or see 'hg help push' for details about pushing new heads)

Oops! Bob has won the race to push first to public. So Alice needs to integrate with Bob: let’s pull his changeset(s) and see what the branch heads are.

$ hg pull ../public
[...]
added 1 changesets with 1 changes to 1 files (+1 heads)
(run 'hg heads' to see heads, 'hg merge' to merge)
$ hg log -G -q -r 'head()' --template '{rev}:{node|short}  ({author})\n'
o  4:9d21d673314a  (bob)
|
| @  3:3363442626b3  (alice)
|/

We’ll assume Alice and Bob are perfectly comfortable with rebasing changesets. (After all, they’re already using mutable history in the form of amend.) So Alice rebases her changeset on top of Bob’s and publishes the result:

$ hg rebase -d 4
$ hg push ../public
[...]
added 1 changesets with 1 changes to 1 files
2 new obsolescence markers
$ hg push ../review
[...]
added 1 changesets with 0 changes to 1 files
1 new obsolescence markers
obsoleted 1 changesets
updating bookmark bug15

The result, in both review and public repositories, is shown in figure 8.

Figure 8 a: review shows v1 and v2 of Alice’s fix, then v1, v2, v3 of Bob’s feature, finally Alice’s fix rebased onto Bob’s

Figure 8 b: public just shows the final public version of each changeset

Setting up¶

For these examples, we’re going to use a slightly different workflow: as before, Alice and Bob share a public repository. But this time there is no review repository. Instead, Alice and Bob put on their cowboy hats, throw good practice to the wind, and pull directly from each other’s working repositories.

So we throw away everything except public and reclone:

$ rm -rf review alice bob
$ hg clone public alice
updating to branch default
2 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ hg clone public bob
updating to branch default
2 files updated, 0 files merged, 0 files removed, 0 files unresolved

Once again we have to configure their repositories: enable evolve and (since Alice and Bob will be pulling directly from each other) make their repositories non-publishing. Edit Alice’s configuration:

$ hg -R alice config --edit --local

and add

[extensions]
rebase =
evolve =

[phases]
publish = false

Then edit Bob’s repository configuration:

$ hg -R bob config --edit --local

and add the same text.

Example 6: Content-divergent changesets¶

When an obsolete changeset has two successors, those successors are content-divergent. One way to get into such a situation is by failing to communicate with your teammates. Let’s see how that might happen.

First, we’ll have Bob commit a bug fix that could still be improved:

$ cd bob
$ echo 'pretty good fix' >> file1
$ hg commit -u bob -m 'fix bug 24 (v1)'                   # rev 4:b2be

Since Alice and Bob are now in cowboy mode, Alice pulls Bob’s draft changeset and amends it herself.

$ cd ../alice
$ hg pull -u ../bob
[...]
added 1 changesets with 1 changes to 1 files
$ echo 'better fix (alice)' >> file1
$ hg amend -u alice -m 'fix bug 24 (v2 by alice)'

But Bob has no idea that Alice just did this. (See how important good communication is?) So he implements a better fix of his own:

$ cd ../bob
$ echo 'better fix (bob)' >> file1
$ hg amend -u bob -m 'fix bug 24 (v2 by bob)'             # rev 5:541f

At this point, the divergence exists, but only in theory: Bob’s original changeset, 4:b2be, is obsolete and has two successors. But those successors are in different repositories, so the trouble is not visible to anyone yet. It will be as soon as Bob pulls from Alice’s repository (or vice-versa).

$ hg pull ../alice
[...]
added 1 changesets with 1 changes to 1 files (+1 heads)
1 new obsolescence markers
2 new content-divergent changesets
new changesets e3a586fd2377 (1 drafts)
(run 'hg heads' to see heads, 'hg merge' to merge)

Figure 9 shows the situation in Bob’s repository.

Figure 9: Bob’s repo with 2 heads for the 2 content-divergent changesets, 5:541f and 6:e3a5; wc is at 5:541f; both are successors of obsolete 4:b2be, hence divergence

Now we need to get out of trouble. As usual, the answer is to evolve history.

$ HGMERGE=internal:other hg evolve
merge:[5] fix bug 24 (v2 by bob)
with: [6] fix bug 24 (v2 by alice)
base: [4] fix bug 24 (v1)
0 files updated, 1 files merged, 0 files removed, 0 files unresolved

Figure 10 shows how Bob’s repository looks now.

Figure 10: only one visible head, 7:aa82, successor to hidden 5:541f and 6:e3a5

We carefully dodged a merge conflict by specifying a merge tool (internal:other) that will take Alice’s changes over Bob’s. (You might wonder why Bob wouldn’t prefer his own changes by using internal:local. He’s avoiding a bug in evolve that occurs when evolving content-divergent changesets using internal:local.)

# XXX this link does not work .. bug: https://bitbucket.org/marmoute/mutable-history/issue/48/

** STOP HERE: WORK IN PROGRESS **

Phase-divergence: when a rewritten changeset is made public¶

If Alice and Bob are collaborating on some mutable changesets, it’s possible to get into a situation where an otherwise worthwhile changeset cannot be pushed to the public repository; it is phase-divergent with another changeset that was made public first. Let’s demonstrate one way this could happen.

It starts with Alice committing a bug fix. Right now, we don’t yet know if this bug fix is good enough to push to the public repository, but it’s good enough for Alice to commit.

$ cd alice
$ echo 'fix' > file2
$ hg commit -A -m 'fix bug 15'
adding file2

Now Bob has a bad idea: he decides to pull whatever Alice is working on and tweak her bug fix to his taste:

$ cd ../bob
$ hg pull -u ../alice
[...]
added 1 changesets with 1 changes to 1 files
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ echo 'Fix.' > file2
$ hg amend -A -m 'fix bug 15 (amended)'

(Note the lack of communication between Alice and Bob. Failing to communicate with your colleagues is a good way to get into trouble. Nevertheless, evolve can usually sort things out, as we will see.)

[figure SG06: Bob’s repo with one amendment]

After some testing, Alice realizes her bug fix is just fine as it is: no need for further polishing and amending, this changeset is ready to publish.

$ cd ../alice
$ hg push
[...]
added 1 changesets with 1 changes to 1 files

This introduces a contradiction: in Bob’s repository, changeset 2:e011 (his copy of Alice’s fix) is obsolete, since Bob amended it. But in Alice’s repository (and the public repository), that changeset is public: it is immutable, carved in stone for all eternity. No changeset can be both obsolete and public, so Bob is in for a surprise the next time he pulls from public:

$ cd ../bob
$ hg pull -q -u
1 new phase-divergent changesets

Figure 7 shows what just happened to Bob’s repository: changeset 2:e011 is now public, so it can’t be obsolete. When that changeset was obsolete, it made perfect sense for it to have a successor, namely Bob’s amendment of Alice’s fix (changeset 4:fe88). But it’s illogical for a public changeset to have a successor, so 4:fe88 is unstable: it has become phase-divergent.

[figure SG07: 2:e011 now public not obsolete, 4:fe88 now phase-divergent]

As usual when there’s trouble in your repository, the solution is to evolve it:

$ hg evolve

Figure 8 illustrates Bob’s repository after evolving away the phase-divergent changeset. Ignoring the obsolete changesets, Bob now has a nice, clean, simple history. His amendment of Alice’s bug fix lives on, as changeset 5:227d—albeit with a software-generated commit message. (Bob should probably amend that changeset to improve the commit message.) But the important thing is that his repository no longer has any unstable changesets, thanks to evolve.

[figure SG08: 5:227d is new, formerly phase-divergent changeset 4:fe88 now hidden]