Version Control, DevOps and Agile Development with Plastic SCM

I always imagine the cycle like this:

As you can see, this perfect workflow aligns perfectly with the DevOps spirit. It is all about shortening the task cycle times (in Kanban style), putting new stuff in production as soon as possible, deploying new software as a strongly rooted part of your daily routine, and not an event as it still is for many teams.

Our secret sauce for all these is the cycle I described above. That can be stated as follows: The recipe to safely deliver to production continuously: short-lived task branches (16h or less) that are reviewed and tested before being merged to main.

How does all the above translate to version control? The "One task - one branch" chapter is dedicated to this, but let’s see a quick overview.

I’ll start after you create a task for a new work to be done, and you create a new branch for it:

Notice that we recommend a straightforward branch naming convention: a prefix (task in the example) followed by the task number in the issue tracker. Thus, you keep full traceability of changes, and it is as simple as it can be. Remember, Plastic branches can have a description too, so there is room to explain the task. I like to copy the task headline from the issue tracker and put it there, although we have integrations with most issue trackers that will simply do that automatically if you set it up.

Now, you check in the changes you make to implement your task. Your task will look as follows (if it is a long task taking a few hours, you should have way more than a couple of checkins). Notice that, the main branch has also evolved (I’m just doing this to make the scenario more realistic).

After more work, your task is completed in terms of development. You mark it as solved with an attribute (status) set to the task. Alternatively, you can mark it as completed in your issue tracker. It all depends on your particular toolset and how you will actually end up implementing the workflow.

Now it’s the reviewer’s turn to look at your code and see if they can spot any bugs or something in your coding style or particular design that should be changed.

As I described earlier, once the review is ok, a peers could do a validation too.

If everything looks good, the automatic part kicks in; your code merges and is submitted to the CI system to build and test.

Two important things to note here:

Then the CI will take its time to build and pass tests depending on how fast your tests are. The faster, the better!

If tests pass, the merge will be checked in (confirmed), and your task branch will look something like this:

Notice the status is now set to merged. Instead of an attribute set to the branch, you could track the status in your issue tracker (or on both, as we do internally).

If the new release is ready to be deployed, the new changeset on main is labeled as such and the software deployed to production (or published to our website as we do with Plastic SCM itself):

By now, you must have a bunch of questions:

You might be ahead of the game I described above. If that’s the case, we still hope you can fit Plastic into your daily work.

If that’s not the case, if you think this is a good workflow to strive for, then the rest of the book will help you figure out how to implement it with Plastic SCM.

Nope! Plastic is extremely flexible and you’re free to implement any pattern you want.

But, as I mentioned in the introduction, I’m going to be prescriptive and recommend what we truly believe to work.

We strongly trust task branches because they blend very well with modern patterns like trunk based development, but we’ll cover other strategies in the book.

For instance, many game teams prefer working on a single branch, always checking in the main branch (especially artists), which is fine.

You have other version control options, mainly Git and Perforce.

You can go for bare Git, choose one of the surrounding Git offerings from corporations like Microsoft and Atlassian, or select Perforce. Of course, you can alternatively stick to SVN, CVS, TFS, Mercurial, or ClearCase…​ but we usually only find users moving away from these systems, not into them anymore.

Why would you choose Plastic instead of the empire of Git or the well-established Perforce (especially since they are in the gaming industry)?

Plastic is a unique software that high-performance teams who want to get the best out of version control use. It’s for teams that need to do things differently than their competitors who have already gone to one of the mainstream alternatives.

What does this mean exactly?

The list goes on and on, but you get the idea.

If Git is eating up the world of software development, who is selecting a niche player as Plastic?

Plastic is used by a wide range of teams: Companies with 3000+ developers, small shops with just two programmers starting up the next big game, and the entire range in between. Teams with 20-50 are very common.

If you are just joining an existing project, you’ll probably just need the client.

Congrats! You are the champion of version control of your team now. Well, we’ll help you move forward. Here are a few hints:

Follow the installation and configuration guide to get detailed instructions of how to install Plastic on different platforms.

This view is like a Windows Explorer or macOS Finder with version control information columns.

You can see the files you have loaded into your workspace and the date when they were written to disk, and also the changeset and branch where they were created (although I’m not including these last two in the figure above).

It is the equivalent of running a cm ls command.

The most exciting things about the Workspace Explorer are:

Actions are enabled depending on the context (for example, you can’t check the history of a private file).

This view is so useful! You can perform most of your daily operations from here. In fact, I strongly recommend using this view and not the Workspace Explorer to add files and check in changes.

The layout of Pending Changes in all our GUIs looks like this:

The other key thing about Pending Changes is the ability to display the diffs of a selected file:

Plastic comes with its own built-in diff tool. And as you can see, it can do amazing things like tracking methods that were moved. We are very proud of our SemanticMerge technology.

Most of the Plastic SCM actions you will do while developing are done from Pending Changes. You simply switch to Plastic, see the changes, check in, and get back to work!

The Branch Explorer diagram is our crown jewel. We are super proud of it and the way it can render the evolution of the repos. The Branch Explorer (a.k.a. BrEx) view has the following layout:

Every circle in the graphic is a changeset. The changesets are inside containers that represent the branches. The diagram evolves from left to right, where changesets on the right are newer than those on the left. The actual Branch Explorer in the GUIs draws columns with dates to clarify this.

The green arrows are merges and the blue ones connect changesets with a "is the parent of" relationship.

The green donuts surrounding changesets are labels (equivalent to Git tags).

The Branch Explorer lets you filter the date from where you want to render the repo, and it has zoom, a search box, and also an action button to show "only relevant" changesets. It also shows a "house icon" in the changeset your workspace is currently on (this is true for regular Plastic, not Gluon, as you will see in the "Partial Workspaces" section).

By the way, if you click "Only relevant", the Branch Explorer will compact as the following figure shows, since the "relevant changesets" are:

This way, it is very easy to make large diagrams more compact by hiding "non-relevant" changesets.

From the Branch Explorer context menus, you can perform several key actions. You can select every branch, changeset, label and right-click on it to discover their actions.

Typical actions you can perform are:

In my opinion, you can perform all the daily actions just using Pending Changes and the Branch Explorer, and maybe a little bit of the Workspace Explorer. If I had to remove everything except what was absolutely essential, Plastic would run just with these three.

There are other views like the Branches View, which lists branches more traditionally (a list), Changesets View, Attributes, Labels…​ but you can use the Branch Explorer instead for a more visual view.

There will be many private files in the workspace that you won’t need to check in (temporary files, intermediate build files (the .obj), etc). Instead of seeing them repeatedly in the GUI or cm status, you can hide them by adding them to ignore.conf. Learn more about ignore.conf in "Private files and ignore.conf."

If you run cm ci from the command line now, only foo.c will be checked in unless you use the --private modifier or run cm add for bar.c first. Let’s suppose we simply checkin foo.c:

Notice how I added a comment for the checkin with the -c modifier. This is the comment that you will see in the Branch Explorer (and changeset list) associated with the changeset 1 you just created.

You would use the cm diff command to diff your first changeset:

In the above figure, in changeset 1, foo.c is added.

Now, if you diff changeset 2:

The command shows that foo.c is modified on that changeset.

The cm diff command is responsible for diffing both changesets/branches and actual files.

Since the changes made were pretty basic so far, the diffs will be very simple. So let me show you a diff from a branch I have on one of my local repos:

As you can see, the diff is more complex this time, and it shows how I changed three files and moved one of them in the branch main/scm23606.

So far, we only diffed changesets and branches, but it is possible to diff files too. So, I’m going to print the revision ids of the files changed in scm23606 to diff the contents of the files:

Then, I can diff the change made in FileSystem.cs as follows:

Instead of a text diff printed on your console, cm diff launches the diff tool configured for text files. Plastic comes with a built-in diff and merge tool we call Xdiff/Xmerge, and then a GUI like the following displays:

We think GUIs are better tools to show diffs than the typical unified diff displayed by many tools, and that’s why we launch a GUI even when the diff is requested from the command line.

If you really want to see a unified diff printed on the console (because you need to run the diff from a terminal without GUI support, for instance), check our Unix diff from console documentation.

Explore the cm diff help to find all the available options. For example, you can diff branches, individual files, changesets, labels (equivalent to diffing the changeset), or any pair of changesets that you want to compare changes.

Well, it is no secret that we put much more attention into GUIs than the command line when it comes to diffing. This is because we think that 99% of the time, developers will diff using a Plastic GUI instead of the command line.

Diffing changesets and branches is very simple from the Branch Explorer; look at the following figure, where you’ll see how there are diff options in the context menus of branches and changesets (and labels).

Diffing an entire branch is simple. Right-click on it to select it (you’ll see that it changes color) and then select the "diff branch" option.

As the following figure shows, diffing a branch is the equivalent of diffing its head changeset and its parent changeset.

The figure shows how diffing branch task1209 is equivalent to diffing changesets 13 and 23.

You can see the selected changesets are colored in violet; when you multi-select changesets, they are colored differently to highlight the fact that many csets are selected. And, you can right-click and run "diff selected."

The diffs of changesets and branches in the GUI are very powerful GUIs like the following mockup shows:

The changeset/branch diff window first summarizes the changes on a tree with changed, deleted, added, and moved, and then shows the actual side-by-side diff of the selected file.

One of the great advantages of the built-in diff in Plastic is that it can to do semantic diffs. This means that the diff can track moving code fragments for certain languages like C#, C++, C, and Java (and then several community-contributed parsers). For example, in the figure above, you can see how the CreateDirectory method was identified as moved to a different location.

Learn more about what "semantic version control" can do.

This is what our branch explorer looks like at this point with the main branch and task2001. If you remember, the workspace was in task2001, so the first thing we’ll do is switch to the main branch, by selecting the branch, right-clicking on it, and running the "Switch workspace to this branch" action.

Once the switch is complete, the Branch Explorer will look as follows, and we’ll right-click on task2001 to run a "merge from this branch":

The GUI will guide us through the merge process. In this case, there’s no possible conflict since there are no new changesets on the main branch that could conflict with task2001, so all we have to do is checkin the result of the merge. In the Pending Changes view, you’ll see a "merge link" together with the modified foo.c. Something like this:

See "Plastic merge terminology" in the merge chapter to learn what "replaced" means and also the other possible statuses during the merge.

Now, once the checkin is done, the Branch Explorer will reflect that the branch was correctly merged to main as follows:

We performed the entire merge operation from the Branch Explorer, but you can also achieve the same from the Branches view (list of branches) if you prefer.

By the way, it is also possible to merge from the command line. I will repeat the same steps explained for the GUI, but this time, running commands.

First, let’s switch back to the main branch:

And now, let’s run the merge command:

The cm merge command simply does a "dry run" and explains the potential merge conflicts.

To actually perform the merge we rerun with the --merge flag:

Let’s check the status at this point:

As you can see, it says foo.c has been "replaced," same as the GUI did.

A checkin will confirm the merge:

And this way, we completed an entire branch merge from the command line 🙂.

This was just an intro to show you how to perform a basic merge, but there is an entire chapter in this guide about the merge operation where you’ll learn all the details required to become an expert.

Annotating a file is easy from the GUI; Navigate to the Workspace Explorer, locate the file, right-click and select "annotate."

From the command line, run:

You don’t annotate a file but a given revision of a file. For example, in the previous example, annotating the file in changeset 16 is not the same as changeset 14 or 15.

For example, if you annotate changeset 12, all the lines will be attributed to changeset 12 since this was where the file was added. If you annotate 13, all lines will belong to 12 except line 30 marked as created in 13.

If a line is modified several times, the line closer to the destination changeset will be marked as "creator" of the line.

There will be cases where dependencies will be required. Ask yourself three times before taking the dependency for granted.

If tasks need to depend on each other, you need to control the order in which they’ll be integrated. If you are triggering the merges manually (discouraged),ensure there is a note for the build-master or integrator in charge.

If you have an automatic process in place, you’ll need to mark branches somehow so they are not eligible for merge by the CI system or the mergebot.

There will be other cases where you’ll be able to keep tasks independent with a bit of effort.

Very often, when you need to depend on a previous task, you need just some of the changes done there.

If that’s the case, and provided you were careful enough to isolate those changes in an isolated changeset (more on that in the "Checkin often and keep reviewers in mind" section), you can do a cherry pick merge (as I explain in the "Cherry picking" section).

A cherry pick will only bring the changes made in changeset 13, not the entire branch. The branches will continue being primarily independent and much safer to merge independently than before.

Of course, this means you must be careful with checkins, but that’s something we also strongly encourage.

There is a second technique that can help when you add a new file on task1213 and you really need this file in the next task. For example, it can be some code you really need to use. This is a little bit contrived, but it works: Add the file again in task1209. Of course, this only works when you just need something small. If you need everything, copying is definitely not an option.

Now, you probably think I’m crazy: Copy a file manually and checkin again? Isn’t it the original sin that version control solves?

Remember, I’m trying to give you choices to keep tasks independent, which is desirable. And yes, this time, this technique can help.

You’ll be creating a merge conflict for the second branch (an evil twin as we call it) that you’ll need to solve, but tasks will stay independent.

The sequence will be as follows: task1209 will merge without conflict.

Then, when task1213 tries to merge, a conflict is raised:

If you are doing manual merges to the main branch, all you have to do is solve the conflict and checkin the result. So it won’t be that hard.

But, if you are using a CI system or a mergebot in charge of doing the merges, there won’t be a chance to make manual conflict resolution there, so the task will be rejected.

If that happens, you’ll have to follow what we call a rebase cycle: simply merge down from main to task1213 to solve the problem, and then the branch will merge up without conflicts.

Imagine that you have to do an important bug fix or add a new feature, but you need to clean up some code before that.

This is one way to do it:

Then the reviewer comes, diffs the changeset number 145, finds there are 100 modified files, and…​ goes for a coffee or lunch, or wants to leave for the day…​ ouch!

And this is how the traditional way to diff branches provokes context switches, productivity loss, or simply "ok, whatever, let’s approve it."

Let’s try again with a different checkin approach:

This time the developer checked in several times during development. It is beneficial for him because he protected his changes to avoid losing them in the event of a weird crash or something. Multiple checkins can also help when you are debugging or doing performance testing. Instead of commenting code out and going back and going crazy, you create real "checkpoints" you know you can go back later safely if you get lost in the way.

But the reviewer will go and simply diff the entire branch because the individual checkins won’t be relevant for him. And the result will be as demotivating as it was in the previous case: 100+ files to review. Ouch!

Now, let’s follow the rule we all use: checkin with the reviewer in mind. Every checkin has to help the reviewer follow your train of thought, follow your steps to understand how you tackled the task.

Let’s try again:

Now, as a reviewer, you won’t go and diff the entire branch. Instead, you’ll diff changeset by changeset. And, you’ll be following the pre-recorded explanation the author made to clarify each step of the task. You won’t have to find yourself against a bold list of 100+ modified files. Instead, you’ll go step by step.

First, you see 21 files modified, but the comment says it was just about cleaning up some C# usings. The list of 21 files is not terrifying anymore; it is just about removing some easy stuff. You can quickly glance through the files or even skip some of them.

Then, the next 12 files are just about a method extracted to a new class, and the affected callers having to adapt to the new call format. Not an issue either.

Next comes 51 files, but the comment clearly says it is just because a method was renamed. Your colleague is telling you it is a trivial change, probably done thanks to the IDE refactoring capabilities in just a few seconds.

Then, it comes the actual change, the difficult one. Fortunately, it only affects 2 files. You can still spend quite a lot of time on this, really understanding why the change was done and how it works now. But it is just two files. Nothing to do with the setback produced by the initial vision of 100 files changed.

Finally, the last change is a method that has been removed because it is no longer invoked.

Easier, isn’t it?

Yes, I hear you. Isn’t it a lot of work to write readable code? Isn’t it easier and faster to just put things together and never look back? It is not, right? The extra work of writing clean code pays off in the mid-term when someone has to touch and modify it. It pays off even if the "other person touching the code" is your future self coming back just three months later.

Well, the same applies to carefully crafted checkins. Sure, they are more complex than "checkpoint" checkins, but they really (really!) pay off.

And, the best thing is that you get used to working this way. Same as writing readable code becomes a part of your way of working once you get used to it, doing checkins with reviewers in mind just becomes natural.

And, since development is a team sport, you’ll benefit from others doing that too, so everything flows nicely.

If you are already familiar with feature toggles, maybe the entire section was irrelevant for you 🙂.

The question we all ask when confronted with "every task must be ready to deploy" is: what about parts of a bigger functionality that is not yet complete?

Because we already asked for tasks to be independent, which usually requires them to be carefully split. But, we came up with a new requirement: small and deployable, which sounds counter-intuitive.

You have a big feature split into 7 parts that will be converted to tasks and implemented using task branches. How is it possible to deploy Part 4 if everything else is not ready?

It’s effortless: You merge it to the main branch and even deploy it, although it will be hidden. It doesn’t need to be hidden for everyone; maybe you can enable the feature for the dev team or for a small number of early adopters. If nothing really works at the very initial phases, it can be simply hidden for everyone.

What is the advantage then if nobody will see it? So why should it be deployed at all?

Hidden doesn’t mean the new code is not passing tests on every release. When the entire feature is ready to be activated, the individual parts have been tested several times. And exercises in individual exploratory tests. And the integration of the last piece won’t trigger a big-bang merge, just a smaller part going through. And the complete deploy cycle, whatever your actual system might be, will be fully exercised.

It is all about setting up a workflow and sticking to it all the time, not only when things are easy or hard. You stick to a way of working and exercise it often so that those long release nights vanish away!

Eons ago, we tried to do formal reviews. You know, the entire thing with a moderator, a short meeting, a scribe, and the author explaining. We only did it maybe twice. Too heavy. It wasn’t natural, we didn’t really adopt it, and it was always being postponed.

So, we changed our minds and started the "no task is merged if somebody else doesn’t review it" around late 2012.

We suffered an initial slowdown for a couple of weeks, but then we all got used to it.

Code reviews proved to be great to train newcomers on "how things get done here" to keep some coherence in the design and the code simple.

Now, fast forward to the present and the DevOps era. Remember the slogan: a continuous flow of high-quality changes ready to be deployed.

You only achieve actual speed if the code stays ready to change. The keyword is stays. Code rots. It gets worse with time after a few changes here and there.

I always like to share this graphic coming from two classics, Boehm’s "Software Economics" and Beck’s "Test-Driven Development":

This is a nice graphic to help management understand why refactors and code reviews pay off 🙂.

Refactoring is the key practice, and code reviews only add to this effort. With every single finished task, there is a chance to make sure the actual design of the solution is fine, that the code is simple enough, and the style matches the rest of the team, etc.

Reviews are also great if they can stop new bugs before they reach the release. But, in my experience, it only happens if the review is very short. Really difficult or long tasks with many modified files or new ones are incredibly time-consuming and hard to review.

And here is where "checkin for the reviewer" kicks in: everything is much simpler with the developer’s help.

Some useful tips:

It really depends on the team. The key is to have at least one, so every task is code reviewed.

More than one can be great, at the cost of a bigger impact on everyone (context switches, extra work that piles up together with actual coding), and an increase in task cycle time (more people involved to move it forward).

We often have one reviewer, but in complicated tasks, ask more than one colleague to review specific parts they know better. It is not something that happens every day for us, though.

What tools? I hear you asking. Well, we provide a built-in code review system in Plastic, and at the time of writing, we just finished designing a major improvement. So, you can definitely use Plastic’s code review or go for a third party like Crucible.

You can even perform totally manual reviews; I’ve done this several times. I open a Plastic branch and diff changeset by changeset, and simply take notes in the issue tracker, in a new enclosure, of what needs to be changed. It is not super fancy or anything, but it works.

A great book on Exploratory Testing that we all love here: "Explore It!" published by Pragmatic Programmers.

Manual tests done by humans shouldn’t be repetitive tasks. If they are repetitive and follow a script, then they should be automated. Of course, time and budget restrictions prevent this, and many teams still do tons of manual testing, but in an ideal world, those are all executed by test frameworks.

What are Exploratory Tests, then?

Simply put: Exploratory Testing is traditional structured random testing. It gives some rules to organize a manual testing process that basically consists of letting the testers arbitrarily test the software instead of constraining them with a script (which could be automated). For me, exploratory testing is the same for testing as agile methods are to coding; it can look at code and fix, but there’s some order which prevents the chaos from happening.

Exploratory testing is based on well-planned and time-constrained sessions. First, the team plans what needs to be tested in advance and writes a set of charters or test goals. Then, they define what they need to test but not how to test it (you can always give some ideas to help, but you can’t create a script).

There are two important topics here:

Each tester takes a charter and starts testing the assigned part of the software, creating a log called testing session in which they’ll write how long it took to set up the test, how long they were actually testing, writing documentation and also exploring unrelated but valuable pieces of software (you start with something and continue with something else you suspect that can fail).

The testing session will record all the issues (minor problems) and bugs detected.

So, there’s probably nothing special about exploratory (my explanation is very simple too) but it helps to organize a whole testing process; it tells you what you should plan and how (constrained times, normally no more than 2 hours per session), and also what results to expect and how to record them.

We record all the test sessions in our internal wiki and link each issue/bug to our bug tracking system, setting a specific detection method to determine how many issues and bugs were detected using exploratory testing. Linking from the wiki to the issue tracking and back allows for better navigation.

We call it "validation," although it is too big of a name that can invoke the demons of formal validation or something, so I better not go there.

This is how it works: someone gets the feature or bug fix and ensures it does what it should.

It is important to highlight that "validations" are not manual tests. As I said, we are supposed to automate those.

The validation is just trying to figure out if what we did makes sense for the user if it is usable if it really fixes the bug (even if a unit test or GUI test was added).

It is some sort of short Exploratory Test that helps ensure we produce a usable and consistent product. Validation can be as short as 5 minutes (switch to the branch, build the code, run it, set up an example, etc.) or as long as 1-2 hours if the scenarios to test are very complex. But, the usual is just 10-20 minutes each at most.

By the way, sometimes we do longer (1-4h) exploratory testing sessions following what "Explore it!" describes. We generate the reports for each session: Captured bugs, followed up with new tasks, etc. We do it when we have big features, but the actual daily practice is the short validations.

Around 2012 we hired an external team to help us do weekly Exploratory Tests, but it didn’t work as expected.

I mean, they were not finding the kind of issues we expected, and we struggled to find out why.

The problem was that the testing professionals were not used to most of the development tasks related to the daily use of Plastic SCM. We thought we were doing something wrong. But, at the end of the day, in my opinion, you need testers who resemble the kind of users your software will have.

And Plastic SCM users are…​ well, developers. They know how to use Eclipse, Visual Studio, the command line, doing branching, merging, diffing code, creating actual code…​ the sort of things the external testers we hired were not used to, and that’s why we failed.

Nowadays, daily validations and code reviews are part of our day-to-day development, and of course, they are time-consuming, but we considered that they are worth every minute. We try to organize our days to avoid interruptions, so many do the pending reviews and validations early in the morning, then the rest of the day can be spent on development.

Let’s look at a typical project following the serial development pattern, better known as mainline development (not to be confused with trunk development). It just means there’s a single branch where everyone checks in their changes. It’s straightforward to set up and very easy to understand. It’s the way most developers are used to working with tools like Subversion, CVS, SourceSafe, etc.

As you can see in the figure, the project evolves through checkins made on a single branch (the main branch). Every developer does a series of checkins for their changes. Since it’s the central point of collaboration for everyone, developers have to be very careful to avoid breaking the build by doing a checkin that doesn’t build correctly.

In the example figure, we see how Vio creates a new loading form (cset: 123) but makes a mistake, and then she has to fix it in a later check-in (cset:125). It means the build broke between 122 and 125. Every developer updating their workspace in between would have been hit by the bug, and it most likely happened to Borja after he checked in cset:124 and updated his workspace accordingly.

Also, if you look carefully, you’ll see we’re mixing together important changes like the one made on cset:121 (a big change on one of the core queries, which could potentially break the application in weird ways) with safer ones like the typo fixed in cset:122. What does that mean? It means that if we had to release right now, the baseline would not be ready or stable enough.

Let’s see how the same scene looks like using parallel development with the branch-per-task method:

As the picture shows, several branches are involved since every task is now a branch, and there are merge arrows (the green lines) and baselines (circles). We could have created baselines before, but you’ll find it’s much easier to know when to create them by using a branching pattern.

Using this example as a basis, I’ll start going through the problems we can find in serial development, how to fix them with task branches, and why this parallelism is better.

How often do you check in when you’re working on mainline development? I bet you’re very careful before checkig in because you don’t want your co-workers coming to your desk, shouting about the code not building anymore, right?

So, where’s your code when you’re working on a difficult task— something hard to change that will take a few days to complete? Is it under source control? Most likely, it won’t be since you’re reluctant to checkin things that don’t compile, that are not complete, or that simply collide with other changes.

This is a big issue and pretty normal with mainline development; changes are outside version control for long periods, until developers are completely done. The version control tool is used just as a delivery mechanism instead of a fully-fledged VCS.

With the branch-per-task approach, you won’t have this problem: you can check in as often as you want to. In fact, it’s encouraged. To preserve your own development process, this enables you to even tell a story with your checkins, as we covered in the "Checkin for the reviewer" section, to preserve your own development process.

Breaking the build is something widespread when using mainline development. You check in some code that you didn’t test properly, and you end up breaking some tests or even, worse, introducing code that doesn’t compile anymore.

Keeping the main branch pristine is one of the goals of the branch-per-task method. You carefully control everything entering the main branch, so there’s no easy way to break the build accidentally.

Also, keep in mind that the usage of the main branch is totally different from a branch-per-task pattern. Instead of being the single rendezvous point for the entire team, where everyone gets synchronized continuously, and instability can happen, the main branch is now a stable point, with well-known baselines.

When you’re working in mainline mode, it’s not often easy to describe the exact starting point of your working copy.

Let me elaborate. You update your workspace to main at a certain point, as you can see in the following picture. What’s that point? It’s not BL131 because there are a few changes after that. So, if you find an error, is it because of the previous changes or due to the ones you just introduced?

You can easily say, "Well, if you’re using continuous integration, you’ll try to ensure the build is always ok, so whatever you download will be ok." First off, that’s a pretty reactive technique - where you first break the build and later fix it. Secondly, yes, you’re right, but still, what is this configuration? If you update at the indicated point, you’ll be working with an intermediate configuration, something that’s not really well-known - you’ll be shooting a moving target!

Now take a look at the situation using task branches:

As you can see, there’s a big difference. All your tasks start from a well-known point. There’s no more guessing, no more unstable configurations. Every task (branch) has a clear, well-known stable starting point. It really helps you stay focused on your own changes.

Update with trunk-based devel.: I took this entire section from some writing I created years ago, dating almost to the very first days of Plastic. Most of what it says is still true, so it made sense to me to add it for clarity. There is one major shift here: we used to do manual integration long ago, where every single task branch was manually merged to main by an integrator (or build master). Now we automate all that. So how does it change the shooting a moving target problem? Well, now every new single changeset in main is totally trustable because it passed all tests. There are no intermediate integration changesets anymore, so you are fine creating a new branch from each new changeset, no problem. Of course, the full moving target stays true for teams still stuck to checkin to main and done because if that’s the case, they might still be on shifting sands.

The psychological side: As mentioned above, it is crucial to ensure every task starts from a stable point. The reason goes far beyond coordination and preventing bug spreading. It has an impact on productivity. If a test fails in your new task branch after a few checkins, you know it is your stuff. It is not finding how’s to blame. It is about the productivity impact. When something fails, and you don’t know for certain if it worked before, you’ll have to spend minutes fighting your brain that already thinks that the bug was provoked by someone else. Funny but true. Didn’t happen to you? But, if you are certain all tests passed in the base of your branch, you’ll jump straight to fixing the problem you just introduced.

Creating baselines is a best practice. Using the branch-per-task method, baselines become a central part of your daily work. There’s no better way to enforce a best practice than making it an integral part of your workflow.

Automation note: Yes, if you are already using full automation, this point will be more than obvious to you 🙂.

People dealing with dangerous materials work in controlled environments, and usually behind closed doors, to prevent catastrophes from spreading if they ever happen. We can learn a valuable lesson from them.

Look at the following mainline example: Vio introduces a bug, and immediately everyone hitting the main branch will be affected by it. There’s no contention, there’s no prevention, and the actions are entirely reactive. You break the code, and yes, you fix it, but the build should not have been so easily broken in the first place.

Now, let’s take a look at the same situation with task branches. The bug will still be there, but we have a chance to fix it before it ever hits the mainline. There’s contention, and there’s a preventive strategy in place.

Every task needs to be reviewed (and validated optionally) before being merged. But the last step, the actual gatekeeper, is the automated test suite.

That’s why you need a solid test suite to implement useful task branches like it is for trunk-based development and DevOps overall.

Check out this article on The Practical Test Pyramid. It is a concise explanation of the famous Test Pyramid:

A simple rule of thumb: write as many unit tests as you can and as few UI tests as you can.

Want to learn why? I’m going to share our own experience on the matter.

Unit tests should be fast, reliable (unless you really write some messy ones), and capable of testing everything. Yes, if you think you really need an integration test because "it can’t be tested with unit tests," I bet you are wrong. I know I was.

Now, we create more unit tests than ever and less and less GUI and smokes. Of course, some are always needed, but the fewer, the better.

We call them smoke internally, but they are sort of integration tests.

We extended NUnit long ago and created PNUnit (parallel NUnit), and we use it to sync the launch of client and server on different machines.

Tests are written in C# and automate the command line.

Our mistake: Each test takes between 4-15 seconds to run. Too slow. We didn’t realize it at the beginning and created tons of them. Many are still in service.

Test slowness ends up creating the "new versions are an event" problem, which is one of the key ones to solve when implementing true DevOps. Otherwise, new versions or releases are something painful that the entire team will try to postpone. And it all ends up in long nights and weekends. Not the way to go.

Fortunately, smokes are stable and false positives are not common.

We also have a bunch of UI tests. They can detect UI issues and help avoid regressions.

In our case, they need to start the server, wait for it to be ready, and so on. And well, they are the root of all evil.

I mean, unless you write them really, really well, they end up being fragile (the worst that can happen to an automated test suite).

We rewrote many GUI tests over the years. This is how the whole thing evolved:

But, even with coded GUI tests, we were hit by long tests and the unavoidable issue of "control not found" that comes with all UI Automation packages I know.

That’s why we ended up writing our own small framework for GUI tests, but that’s a whole story in itself.

If you are reading this and your team already has a solid test suite with great test coverage, then you are all set.

If not, let me share a trick: start small. I’ve seen so many teams being discouraged by their total lack of tests. "We’ll never get there," they thought. But, of course, you can’t pass from zero-tests to 10 thousand unit-tests by stopping all development and just focusing on testing for months. That’s not realistic. Business must go on. So, what can you do? Battle inertia. The easiest is to simply continue with zero tests. Write one today, have a few in a week. Keep adding. In a month, you have a small number. In a year, the code will be quite well tested. The alternative is to sit exactly where you are now in a year, which I bet is not such a good choice.

I like to think of tests as a safety net that gets thicker and thicker as you keep adding more.

The following picture shows what I’m trying to explain:

The "safety net" example also gives you a good idea for the strategy to follow: focus on catching big issues first.

A couple of rules we follow:

Both techniques help even if you don’t have a test suite in place; they’ll help move things forward.

Finally, there will be cases where you need to pass an additional test suite before publishing. Again, t is not ideal, but slow test suites or the need to run load tests for hours before deploying the new version might force you to do that.

If that’s the case, the situation will look as follows:

Teams can perfectly label each new build, but sometimes it is better to save labels for real versions ready to deploy and use attributes for build numbers. We use this method ourselves when releasing Plastic. Every new changeset in the main branch has a build number, but only real stable releases have labels.

As you can see, build BL132 is used to launch tests grouping the new tasks merged from 129 to 132.

While the test suite passes, new tasks can reach the main branch:

Then, after the release test suite passes, the new version is labeled:

Consider this a huge warning: This practice will create broken builds. The task test suite won’t be complete enough because it requires a second test suite to pass, so the second one can detect issues that prevent the new version from being released. If that happens, you are in a broken build. So you’ll need a new task to fix it, and it must reach the main branch as quickly as possible to revert the broken build status.

The situation, is not as bad as breaking the build with every task because your main branch might not be good enough to go public for everyone. However, it can be still be deployed to smaller groups or even to the internal development team, which is good for dogfooding.

It is a technique to ensure that all changes reach trunk quickly, and every change is correctly reviewed and tested. You can find out more about it on Hammant’s super site. Trunk is the main development line, "main" in Plastic jargon.

Trunk-based development is the foundation of "continuous delivery."

It requires continuous integration; changes get continuously merged into trunk as soon as they are known to be good (peer-reviewed and tested with an automated test suite).

Trunk-based development is not the same as mainline development. Remember the old days with everyone just doing checkins to the main branch? (SVN anyone?). No, it is not the same thing. Trunk is all about making sure the build is stable, ready to be deployed. Nothing to do with the old mess.

Before you cringe, trunk-based development is compatible with short-lived task branches.

I wouldn’t even call these branches "feature branches" at all because it might be misleading. A feature can take a long time to implement, so the branch grows too old.

I prefer "task branches," which are sometimes used interchangeably, although the meaning is quite different.

Tasks are short, or at least they should be. How short? 4h, 8h, ideally no longer than 16hours.

Features are longer than that, so we just split them up. Task branches are great for that purpose, and they blend nicely with trunk-based development.

Well, suppose you are working in a distributed environment. You can checkin locally and then push, can’t you? After all, this ends up being a "local branch," and you have a place to checkin as frequently as you need before hitting "main." You can do this with Plastic when you work distributed, of course.

Now, why should it be different if you are working centralized? You can create branches, checkin often then set the branch as ready to be merged to main.

In both cases (distributed and centralized), short task branches bring lots of benefits:

It is not my intention to go under the hood and into the internals here, but just take a quick look into how repos actually store. Since chances are you have a strong technical background, I’m sure you’ll appreciate understanding how things work. (If not, feel free to skip this, I don’t want to bore you with the gory details).

We have two main types of storage in Plastic SCM. They are completely transparent outside the server. I mean, when you are doing a checkin, or merging, or doing diffs, or even running a replica, you don’t really care whether your server is storing repos in SQL Server or Jet. In fact, servers talking with each other don’t care either. Client-server and server-server communication are totally independent of the storage backend.

That being said, the two major storage types are:

If you want to learn more about the story of Jet and why we finally reinvented the wheel, read the story of Jet.

This is a mantra in software development. Try to keep things as simple as possible. For example, ask yourself several times if you really need to create a component repo.

Overdesign will normally bite you harder than a too simplistic approach.

I have to admit that before reading about monorepos as a legit approach, I always thought putting everything on the same repo was a little bit of a mess.

But, let me share how we structure our main repo to illustrate how we implemented monorepos internally.

We work on one main product: Plastic SCM. But at the time of writing this book, we have two other secondary products: SemanticMerge and gmaster.

We manage the three on a single repo called codice, as you can see below:

We could have certainly structured it differently with Xlinks as follows:

But since both gmaster and SemanticMerge require pieces inside 01plastic/src/client to build, the two new repos wouldn’t build in isolation. We can admit that, they only build with mounted inside codice. But, if that’s the case, what’s the point in isolating them? Plastic doesn’t really have repo size issues, so the division would be quite artificial and impractical.

Another alternative would be to split the repositories differently, so both SemanticMerge and gmaster can be built separately from the main codice repo. For example, a stricter separation in components would be as follows:

I marked web inside gmaster in green because this could also be an opportunity for another Xlinked repo.

We can continue our splitting flurry, but the question is: does it really make sense? Plastic can certainly cope with all those repos but, does it really make sense for the team?

In our case, the answer was no.

Everything was simpler with a single code repo. Even the website for gmaster is on the same repo because this way, it is very easy to build the same REST API definition both in the client and the server and refactor accordingly instead of splitting. It could be a good candidate for a different repo but, why?

I don’t mean it is good to have a big mess in the repo. We are super strict with directory and file naming, and with what goes into each directory and so on, and we frequently reorganize directories when things don’t look nice anymore. But a single repo serves us well.

I explained my view about how important is to keep things simple, but of course, there are cases where splitting and xlinking is really the best option.

How can we distinguish the scenarios?

It all depends on practical and organizational restrictions.

If you’re familiar with Git and Perforce or SVN, you know what distributed vs. centralized means. If not, check the following explanation.

Centralized means you directly checkin to a repository (repo) located in a central server, typically on-premises.

Subversion, TFS, and Perforce are typical examples of the centralized workflow. Plastic can work in this mode too.

Distributed means you always checkin to a local repo in your machine, then push your changes to a remote repo on a server.

Distributed is great because every checkin is local, so it is super-fast (no network latency potentially biting you). The downside is that you need two operations to reach the central repo: checkin and then push. This is not a big issue, of course, considering the benefits, but our experience tells us that it is like torture for certain profiles. In video games, for instance, artists hate push and pull. Checkin and done is more than enough for them.

We chose the name Plastic because we wanted to be flexible (yes, there are rigid plastics 😉). So, Plastic can be as fully distributed as Git is or work fully centralized.

To work distributed, we need to provide local repos, and that’s why Plastic runs a super lightweight local server. It is the same codebase that can scale up to manage thousands of concurrent users and millions of requests per day but tuned down to be as quiet and resource-constrained as possible.

There is some confusion about this that I think is worth covering. Some developers wrongly believe that you need a distributed version control to do proper branching and merging.

That’s why only a few version controls were good with branches before 2005. That was the year when Plastic SCM, Git, and Mercurial were born. All of them are distributed, and when Git got worldwide popularity, everyone thought good branching was a DVCS thing.

I’m a version control geek, so I must say that while SVN, CVS, Visual Source Safe, Perforce, and a few others were truly weak with branches and merges, Clearcase had a pretty high bar. It was very good, but it was hard to install, extremely expensive, and after the IBM acquisition, it got more bad press than good. But it was one of the strongest systems ever made.

Getting back to the point, you can do super-strong branching and merging with Plastic while working centralized.

You can install your own Plastic SCM server at the office. This will be an on-premises server. This might sound obvious, but I think it is worth making it crystal clear, especially for non-native English speakers 🙂.

Then, we offer a service to host repositories in the cloud, managed by us, so your team is freed from the sysadmin burden. So that’s the Cloud option.

What impact is there when choosing Plastic Cloud vs. on-premises on the distributed/centralized strategy?

None! You can work perfectly centralized or distributed with Cloud or on-premises. The only downside you might have is the added network latency if you compare Cloud with a server hosted in your office.

Something you must keep in mind when designing the actual version control layout for your company is that Plastic gives you a range of choices. You don’t have to force the entire team to stick to a single pattern. Some can work centralized, some distributed, some connected to the cloud, and you can even create multi-site setups.

The following diagram is an example of a possible setup:

Multi-site stands for having multiple on-site servers connected through replicas. (Important: We call replica to push/pull, nothing new.)

The scenario can be as follows:

Multi-site was an essential concept before distributed version controls were widely adopted, as almost the only way to connect different sites.

My recommended layout is the same I made for branching patterns and repo layouts: Try to keep things simple, from an IT perspective and for the team members using Plastic daily.

What I mean by this is:

In short, Plastic gives you all the flexibility, but it doesn’t mean you have to use everything 🙂.

You have another choice to connect distant sites: the cache server, a.k.a. proxy server.

It is a straightforward piece of software; it caches data (not metadata), so it’s not downloaded from the distant server each time, saving network bandwidth and time.

The following diagram explains how the Plastic clients (GUI or command line) and a proxy/cache server work:

As you can see, the functionality is straightforward.

The typical scenario is a co-located team that usually works with big files and is connected to a distant server.

The team at Boecillo will benefit from faster updates (data downloads) if they have a proxy on-site because data won’t be downloaded all the time from Boston.

The scenario works well as long as the network connection works as expected. Proxies are a performance improvement, not a solution for multi-site. It is important to keep this in mind when you design your Plastic server infrastructure.

The downside with proxies is that if the network connection between the cache server and the central server goes down, the users won’t operate normally. For example, they won’t be able to checkin anymore, and updates will also fail because only data is cached, not metadata.

Having your repos on your machine or your on-site server for the entire team (fully distributed for the first option, multi-site for the second) allows you to have a solution resistant to network failures. The team can continue working even if the connection to the central server goes down.

If distributed and push/pull operation (a.k.a. replica) is superior to proxies, why do we support them?

In short, we strongly recommend using distributed if possible. Still, proxies are always an option when replicas are not possible.

It is important to note that some teams, especially those coming from Perforce, tend to prefer proxies over distributed because that is what their previous version control provided as a multi-site solution. So, please, keep in mind the options and remember Plastic can do distributed when needed.

When you look in a Plastic repository, you’ll see that changesets are numbered as follows:

A changeset zero is automatically created together with the main branch when you create a new repo.

Then, as soon as your checkin changes, there will be other changesets, in my example, changeset 1 and 2.

But, these changeset numbers are local to the repository. I mean, they are good if you work centralized, they are meaningful and easy to remember numbers.

But as soon as you work distributed, local numbers are no longer suitable, and you need Globally Unique IDentifiers – GUIDs.

So, if you look carefully in your Plastic SCM Branch Explorer, you’ll see there is always a GUID number for each changeset.

Thus, changesets are identified by two numbers: locally by numbers and globally by a GUID.

The GUID is a long identifier, so in short, I’ll be referring to it in a shortened way just by its first section. This shorter section is not guaranteed to be unique, but it is usually good enough to refer to it in examples. And chances are, you can even use it to refer to a changeset globally to a team member since chances of a collision are small.

With that in mind, the previous diagram can be simplified as follows:

A push takes changes from one repository and pushes them to a different one. The repos can be on the same server, but they are typically on separate ones.

Let’s follow a simple example step-by-step, so we can ensure you get a solid understanding of how it works.

First, I start with a repo created on a server called juno. Fun fact juno used to be our on-premises main server years ago 🙂. By the way, instead of juno, it could be a local server, something like quake@local for Cloud Edition users, or quake@localhost:8087 if you have a local Team server on your machine. The actual operation will be the same.

As you can see, my quake repo has three changesets.

There is a different repo hosted in my Cloud organization: quake@gamecorp@cloud. Again, instead of a hosted repo in Plastic Cloud, it could be something like quake@skull:9095 for an on-premises server.

Notice how the changeset 0 has the same GUID in both repos. This is intentional. When we designed Plastic, we wanted it so that all repos had common ground so they could all replicate each other - more on this in a second.

Now, what if you now decide to push from main@quake@juno:9095 into quake@gamecorp@cloud?

The first thing Plastic has to do is to determine what needs to be pushed.

To do that, server juno:9095 connects to gamecorp@cloud and retrieves its changesets on the main branch. Then, it finds there is a common point: the changeset 0, and that the two "children" of 0 —`fc113a1e` and 3eef3b52— are the ones that need to be pushed to complete the replica.

Then, the source server juno:9095 (source of the push) will package the two changesets and send them to gamecorp@cloud, together with its corresponding data. I mean, it will send all the needed data and metadata.

Once the operation completes, the situation will be as follows:

At this point, the two repos will be identical.

Now, what happens if you checkin a new change to quake@juno:9095 and want to push again? Well, the same calculation will happen again:

This time, the server juno:9095 determines that 5c1e3272 will be the changeset that needs to be pushed because it found that the remote server already has 3eef3b52.

And this is how push works. It’s very straightforward.

Let me go back again to the changeset numbering problem and the GUIDs. Let’s start with the example above: There is a new change 5c1e3272 to push between juno and gamecorp@cloud. But, a new branch was created in gamecorp@cloud.

Note that I’m starting to color changesets based on their origin: One color for the ones created at juno, a different one for those originally created at gamecorp@cloud.

The new changeset on branch task1273 at gamecorp@cloud has changeset number 3. It collides with changeset 3 created on juno, although they are obviously different.

Locally, each repo tries to follow a numbering sequence, and that’s why numbers can collide.

Check how the two repos look like once the new push operation is completed:

Changeset 5c1e3272 is local number 3 in juno, but number 4 in gamecorp@cloud.

That’s why you can only trust changeset numbers locally but need to use GUIDs if you work distributed.

I hope this explanation makes it obvious why we use the two identifiers.

I’m going to explain the difference between a push and a pull operation in Plastic.

The obvious is true: Push means sending changes from A to B. Pull is bringing changes from B to A.

In Plastic, you can push/pull repos hosted locally and repos hosted on different servers.

The following is a typical push/pull schema between a local repo (hosted on the server called local) and an on-premises server called juno.

When you ask your local server to push to juno, what happens is something as follows:

Your Plastic client connects to the local server and asks it to communicate with juno and negotiate the push. Then the steps described in how push works happen.

Now, when you pull from juno, you ask the local server to get changes from juno. Again, your client connects to the local server, then the local server connects to juno and negotiates the pull. The pull is quite similar to the push, but instead of sending changes, it simply performs a negotiation to download data and metadata.

But, remember that in Plastic, you can perform push and pull operations not only from local repos, like in Git, but also connect to remote repos.

Thus, the following diagram shows a pull operation pulling changes from juno into a new server called skull.

You first ask skull to pull from juno. Then skull connects to juno, negotiates the pull, and then downloads the metadata and the corresponding data.

Think for a moment about what I described. Unlike what happens in Git, you can trigger replicas from any server, not only from your local repos.

Then, in Plastic, you can achieve the same behavior in different ways: Pull from juno into skull is equivalent to a push from juno to skull.

The following diagrams illustrate the equivalent scenarios and highlight a key point: The operations are equivalent but not the required network connections.

When you pull from juno into skull, skull is the one connecting to juno. The diagram shows the direction of the connection and how the data is transferred.

The important thing to note here is that while the two previous operations are equivalent in data transfer, the required network connections are different. skull might be able to connect to juno, but the opposite might not be true, so the second alternative, pushing from juno to skull might be forbidden due to network restrictions.

The following diagram explores a new equivalence. A push from skull to juno is the same as a pull made by juno connecting to skull. But again, the network connections are different and the restrictions might disallow some combinations.

Hopefully I made this crystal clear. It is not difficult at all, but we often find users confused about the equivalences of push and pull between servers. That’s why I created a new summary diagram:

Finally, for the users that are going to use Cloud Edition or Cloud extension, look at the following:

Now, it’s time to find out how to handle changes that happen concurrently. Let’s start with the following scenario. We initially started with a quake repo in juno that we pushed to gamecorp@cloud. Now, developers created changesets concurrently on both sides. As you can see, there is a new changeset 5c1e3272 on quake@juno and two new ones at gamecorp@cloud.

If you try to push from juno to cloud now, Plastic will complain, saying it can’t complete the push because it would create a multi-head on destination.

What in the world is a multi-head?

If the push were allowed, your cloud repo would end up like this:

Where both fb4f9ebc and 5c1e3272 would be heads. If that happens, which would be the changeset at the head of the branch? What would be downloaded when someone tries to update from main? Which changeset would be the selected one?

That’s why Plastic prevents the creation of multi-heads on push because it asks you to fix that situation in your source repo before you push.

That’s why you will need to pull from Cloud before pushing your changes. Once you pull, the repo at juno will be as follows:

The two changesets created on Cloud will be downloaded and linked correctly. 89e9198d as a child of 3eef3b52.

The problem is that you are again in a multi-head scenario. In reality, the old head cset number 3 (5c1e3272) will still be the official head of the branch main at juno, but if you now try to push to Cloud, Plastic won’t let you complete the operation, warning you that you have two heads.

The update will still work correctly downloading cset 3.

What is the solution to this situation? We need to merge 3 and 5 to create a new changeset that will be the single head of main on juno, and then we’ll be allowed to push.

The figure shows how the new changeset b2667591 combines the two old heads and creates a unified head. Now the branch is ready to be pushed to Cloud.

Once the push to Cloud is complete, the result will be as follows:

As you can see, the final result, changeset b2667591, means that at this point, the two repositories are equivalent, although their evolution was different.

When you work with a single central server, it is obvious which one is the source of truth: The repos on the server. Even if everybody has their repo with a clone or a partial replica of one of the repo and work in push/pull, everyone knows where the master is.

This is the model that most of the DVCS follow. This is what GitHub is all about: A central solution for a distributed version control (beyond many other things like an incredible social network for programmers).

Now, if we go for a slightly more complex deployment, with several sites collaborating, is it still easy to know where the source of truth is?

In this scenario, while there are two sites with their servers, it might be easy to propose the Cloud as the rendezvous point where all changesets and branches are pushed. It might ideally be the single source of truth.

Now, what if we remove Cloud from the picture in the previous scenario? The result would be as follows:

Now, it is unclear which one is the leader and which one is the follower.

Of course, it can be immediately resolved by a company-wide agreement: Riga is the main server, everything must be pushed to Riga. This doesn’t happen often by consensus but due to historical reasons: The original site rules.

The alternative is shared ownership of the truth which can end up being a very good solution too: London and Riga can share the main versions, but there is probably no need for both to contain all the task branches created at each site. Each one can be the owner of certain repos. But let’s focus on a single one: How is it possible for them to collaborate without both sides being some perfect mirror of the other?

Look at the following diagram:

What you can see from the example is:

I was super careful to merge branches in Riga only after the changes from London were received. This is not realistic, but I did it to keep the scenario simple. Do you see what would have happened otherwise? If new changes were on main in Riga in t2, for instance, then London would have required a pull and merge (and subbranch) before pushing. Not hard, but it would be more complex than the scenario I drew.

Now, what happens in London once a new version is created in Riga? It just needs to pull main:

There are a few interesting takeaways here:

I’ve introduced a couple of choices and the concept of different repos not having to be exact clones. Now I’m going to share what we think makes the most sense.

To me, the previous scenario would have been much simpler this way:

The workflow is not very simple:

Let’s now see what happens to London once the new version is created: It can pull main from Riga to get the new version, and merges of the task100 and task102 branches. Notice how the other branches task103 and task104, do not need to be pulled from Riga at all, and the main branch will still be consistent and equivalent to main@core@riga.

This scenario I just described can happen in a multi-site situation like the one I also described before. But it will also be extremely common with developers working distributed in their repos. Let me explain:

What if you need to have two repos that are exact clones of each other? Is this achievable? And, I mean, identical clones, not just equivalent results, as I mentioned a few times above.

As soon as developers work concurrently on two repos, there is no guarantee to have them as exact clones all the time. They’ll eventually be consistent clones at the instant when all changesets are replicated to both sites, and concurrent changes are merged and the results of the merges propagated. See what I mean? If you can stop working on both sides and just allow work to resolve conflicts on branches, then eventually the repos will be exact clones. But, since development typically never stops, then this eventual consistency tends to be delayed. It is not a big problem, as long as you’re aware of it.

There is an easier way to get exact clones without dealing with concurrent changes: Keep one of the repos read-only except for replicas from the source repo. Suppose no actual development happens on the destination repo. In that case, you can guarantee that they are exact clones, at least until the changeset that was just replicated into the destination.

This is very good if you need a failover repo (or set of repos) on a different server that you can switch to if something goes wrong on the main one or use it as a backup.

Replicas can be fully automated without any manual intervention. If development happens at both repos, then there will be chances of manual conflict resolution during merges. On the other hand, if changes only happen on one side, there will never be a need for manual intervention, and replicas can be easily scripted. The replicas can be launched by a trigger each time a new changeset is created on the source repo, or periodically every few minutes or hours, depending on your needs.

I like to compare the things that work differently in Plastic than Git since many of you will very likely have a Git background when you try Plastic.

The replication process works differently in Git than in Plastic. And these partial replicas are one of the key differences.

Let’s go with a new example, this way checking two identical repos, in Plastic and Git:

So, what happens in each system when you decide to pull the task029 branch?

In Plastic, the two repos involved will negotiate which changesets need to be pulled: Only the cset in task029 will be selected. Of course, it doesn’t mean only the files that were checkin in this cset will be downloaded, but I’ll cover that later.

In Git, a tree walk operation will happen. Considering the topology of the graph I drew, the commits marked in green will be downloaded:

In this case, Git would download the full repo with all the changes. This is because the tree walk goes first to master, then the head of the master branch has two parents, so the tree walk will walk on both paths.

Of course, in Git, there are ways to limit depth, so you don’t end up pulling the entire repo if you don’t want to. But, when you limit depth, then there might be restrictions trying to push new commits back.

I wouldn’t honestly say any of the alternatives is better than the other. Of course, I’m biased towards our design decision for obvious reasons, but it is important to highlight the difference.

Changesets play a double role: They track the actual changes you checkin, and are also a complete snapshot of the entire repo at the instant of the checkin.

This is important for you to understand partial replicas.

As usual, I will use a concrete example and a bunch of pictures to explain it.

The changesets in the diagram are decorated with the actual changes.

This explains what each changeset contains in terms of changes. But what are the actual trees they point to?

This is the tree for changeset 1, the one adding the three files. As you can see, the changeset points to a specific revision of the root directory, which points to the rest of the revisions of the directories and files it contains.

Now, the changeset 2 in branch task001 modifies foo.c. This is what its tree looks like:

This is how it works:

In the figure above, green indicates the revisions created in changeset 1, dark grey shows the revisions linked to tree 2 but were not created there.Finally, light grey indicates the revisions that are no longer pointed by changeset 2.

The following graphic shows the trees of the next changesets: 3, 4, 5, and 6. Instead of drawing crossed lines all around, I just drew the references to the pre-existing subtrees in every new changeset tree.

Now, let’s go back to the replication scenario. When you pull task002, you won’t be just obtaining foo.h, but the entire tree changeset 6 points to. It is an entire snapshot of the project, so it is fully operational.

If you pull task002 you won’t get the entire history of foo.h, foo.c, and the rest of files. Instead, you’ll get the actual revision required to load the tree of changeset 6.

In contrast, if you replicate main, you’ll be replicating all the changesets in the branch: 0, 1, 4, and 5, and with them will come all the individual trees of these changesets, and a good part of the history of the files involved (excluding the changes made in task001, unless you decide to replicate that branch too).

Partial replicas allow us to replicate just one branch. But what if you want to replicate one changeset of a branch and not the entire history? As you saw in the previous topic, it is technically possible.

When writing this, we didn’t have an interface to replicate only changeset 4 of main. This is not because of any technical restriction, we could easily do it, but we didn’t add a way to specify just a single changeset in a pull.

Until we implement that, there is a trick you can use. Suppose you want to replicate changeset 4 (and, of course, its entire working tree) instead of the entire main branch. Just create a branch from4 as I did in the diagram, and replicate it. You’ll get just the cset 4 replicated to your new repo.

Why is this useful? In our case, our main repo has been evolving since 2006. So, if I want to replicate main to my laptop, I’ll be downloading a good number of gigabytes. That’s not a problem at all, but suppose I want to save a bunch of gigs because my SSD is running out of space. Even if the repo is large, the working copy in the main head is just 500 megs or so. So, I find it very useful to have the option to replicate just the equivalent of a working copy and be able to evolve from there locally, instead of having to download the whole history of the main branch.

I can always connect to the central repo to browse the entire history, while my local stays tiny.

Like we mentioned while talking about "sources of truth," we must be careful when running certain merges on a partially replicated repo. This is because some missing merge history might alter the final merge result in the partial replica.

It is easier seen with an example:

If you merge bug199 to main at Riga, the common ancestor for the merge will be the changeset marked as B in green, which is the correct one.

Now, suppose you only pull bug199 in your local repo and try to merge to main locally. The common ancestor will be changeset A, marked in red. The merge result will be different (possibly more conflicts) than if run in Riga.

That’s another reason why we prefer to do the merges on the central site.

Of course, you are free to merge locally between branches with a simpler hierarchy that you know you own entirely, like a child branch you created in your repo.

Whenever you create a new branch, it is by default created as a child of the one it starts from.

In this case, main/task001 is still empty (and it points to the same tree as changeset 2) since it doesn’t contain any changesets. Its name says that task001 is a child of main. This is because branch objects in Plastic have a parent field pointing to the branch they are child of.

Thanks to this relationship, it is possible to create entire branch hierarchies:

Branch hierarchies are a way to introduce a namespace structure in your branches. They don’t have any other impact other than keeping the branch structure better organized.

In every repo, there is at least one top-level branch: main. But, you can create as many top-level branches as you need. In fact, from a practical perspective, both child and top-level branches have the same behavior.

The following figure shows all branches created as top-level ones.

As I explained above, it is up to you to use top-level or child branches. Their only impact will be how you name the branches and how it is easier to manage your project.

Let me propose a strategy that I find useful: Keep top-level branches for structure branches and child branches for tasks and bug fixes.

Take a look at the following diagram. First, we have the main branch with several task branches. Then, a fix-3.0 that can be a maintenance branch to evolve an older release, and it also has its own set of child branches.

One of the advantages of defining a strict structure of top-level and child branches is that you can filter by level in the Branch Explorer in the Plastic GUI, and then easily focus on the structural view, by filtering out all second-level branches as follows:

This simplification helps you focus on the repo structure by hiding more tactical short-lived branches.

Plastic can also manage branches inside branches. In reality, the changeset structure is the one creating the actual trees of divergent history. With that in mind, it is perfectly possible to have something like the following:

Changesets C1, C2, and C3 all belong to main, so they are a branch inside a branch.

To learn more about subbranches, see "Handling concurrent changes on replicated repos". When creating subbranches, you typically need to make concurrent changes on the same branch in replicated repos.

Branches are created to be merged. However, only a small number of them stay independent, creating parallel dimensions of unmerged code.

Merge is the operation to incorporate the changes made in a branch into a different one.

In the figure, to incorporate the changes A, B, and C done in bug-2001 into main, you need to merge bug-2001 to main.

The result will be something as follows:

Where the resulting changeset R contains the changes made in A+B+C and puts them in main.

The merge from a branch is technically equivalent to the merge from a subbranch since the same logic applies:

Merge is also the operation to merge from a given changeset instead of taking an entire branch.

If you merge from changeset B into main, you’ll be getting the changes made in B and A but skipping C.

Merging bug-2001 is equivalent to merging C (the head of the branch).

There are ways to merge only the changes made in a changeset and not the ones in their parents. This is called cherry pick and we’ll be covering it in detail in this chapter.

All through the book, you have seen a massive number of repository graphs. It is the same graphic Plastic renders in the Branch Explorer. It is a representation of the repository evolution through time. The history of the changesets and their branches, how they evolve, diverge, and converge again.

Each time you checkin, you add a new node to the graph:

The arrow direction shows a "is parent of relationship". This way, 0 is the parent of 1. And 2 is the parent of 3. They might belong to the same branch or different ones, but the important thing is the child to parent links between them.

Finally, the merge links indicate the other type of relationship: Source to the result of a merge.

If we remove the branch decorators, what is left is a typical graph with a single root (changeset 0) and then nodes that diverge in branches and converge again with merges.

This graph is essential for users to understand the repo history, but it is also the fundamental data structure that Plastic SCM uses to resolve merges.

Now, I’ll explain why arrows in the Branch Explorer diagram are rendered the way they are.

We always say the Plastic changeset structure is a DAG: Directed Acyclic Graph. Acyclic? It is full of cycles, thanks to the merge links! Well, visually, yes, but internally for merge tracking calculation purposes, all links go backward.

This is how Plastic walks the tree from 5 back to changeset 0.

The first path is obvious. Simply back from 5 to 2 thanks to the parent link, then back in a straight line to 0.

The second path goes walking back the merge link from 5 to 4, then back.

Why is it then that the merge link is displayed this way? If everything is heading back, why is the merge link the only one going in the wrong direction and creating a cycle?

As I said above, merge links are always walked backward. Always. From result to source. They are never walked in the direction they point to.

Why do we keep them this way?

Well, when we talk about a merge, we typically say: merge changeset 4 to main. We talk about a merge as from a branch or changeset to a destination branch or changeset. That’s why we always considered it very natural to render merges this way.

Of course, there are other possible options, as the following diagram shows:

Should we change how we render the Branch Explorer to option 1 or 2? I think it is worthwhile to understand why everything is rendered the way it is.

Every merge has 3 contributors: The source, the destination, and the base or common ancestor.

Check the following figure:

If you want to merge from bug2061 to main, then the contributors are as follows:

Once the merge has started, the resulting graph will look as follows:

Note how I used dashed lines for the result, the merge link pointing to it, and the line connecting the result with its parent, the destination. This is how we draw in-progress merges. Once the changeset resulting from the merge is checked in, the lines will be uniform and not dashed.

If you decide to merge from main down to main/bug2061, the contributors will be slightly different:

The base would remain intact, but source and destination would be switched. The Result is always a destination child, so it would now belong to main/bug2061 instead.

The cases above are considered the canonical ones where the source and base are separated by at least one changeset.

Let’s see a new scenario:

In this case, you want to merge from main/bug2061 into main, but there weren’t new changes in main after bug2061 branched off. Then, what happens is that the same changeset is simultaneously base and destination. This is typical when you create a couple of branches and then merge the first one back to main. There is no possible conflict because the actual topology of the merge says no concurrent changes exist.

It is essential to note that Plastic always creates a changeset as a result of a merge.

I highlight this because this is radically different than what happens in Git. Since many of you are familiar with Git prior to your jump to Plastic, understanding the difference is worthwhile.

The first scenario is where the destination and the base are not the same changeset/commit. This is a case where conflicts might potentially happen. As you see in the figure, Plastic and Git behave in the same way. Both create a result changeset that reflects what happened.

The second scenario happens when base and destination are the same changeset/commit. In other words, if no possible conflicts can occur due to the repo topology:

Here Git performs a fast-forward. There are no possible conflicts, so it moves the master to the same commit where bug2061 is. It is quite efficient because new commits are not created. The drawback is that the history of the branch is lost, and you can’t figure out anymore which commits were created as part of bug2061 and which ones as part of main.

In Plastic, we always create a result changeset. This is because a changeset can’t be in more than one branch at the same time, and also because it is crucial for us to properly preserve the branch history.

As you saw in the previous section, there are cases where there are no possible conflicts. For example, suppose the base and the destination are the same changeset. In that case, all Plastic has to do is add the correct files to the resulting changeset since no possible conflicts can happen.

The other scenario is when the shape of the graph tells us that there are potential conflicts. I say potentially because, more often than not, there might not be possible conflicts even when the graph has concurrent changes. Let me show you why:

In the figure, you see how in main/bug2061 only inc.c was modified while in main after bug2061 was branched, only foo.c was changed. This means that while, at first sight, there might be potential conflicts in the graph because there are concurrent changes, in reality, no files were changed concurrently.

The same graph topology can lead to conflicts when concurrent changes on the same file happen, as the branch bug3001 shows in the following figure:

Here bar.c was changed both in source and destination, which provokes a file conflict. The conflict can still be automatically solved, as we’ll see in the 3-way merge, but there will be cases where manual user intervention will be necessary.

Look at the following case:

Did you delete line 70, or did I add it?

There is no way to know looking at the two files. Only our knowledge of the actual file would tell.

This is a 2-way merge; you merge 2 files by comparing one to the other.

As you can see, there is no way to merge files with a 2-way merge without manual intervention. Conflict resolution can’t be automated at all.

This is how old version control systems (namely SVN, CVS and several others) used to work, and that’s the primary reason why an entire generation of programmers all around the world developed "merge phobia". Can you imagine having to merge 300 files when all merges need mandatory user intervention, even if changes were trivial?

That’s what a world without common ancestors would look like 🙂.

Let’s forget about version control for a second. Let’s focus on manually solving a conflict like the one in the previous example. How can anyone solve a merge conflict without having previous knowledge about the project?

They would need to check what the file looked like before the two contributor made changes.

This "how the file was" is what we call the base. If we use it to solve the conflict, then the scenario will be as follows:

As you can see, the conflicting line 70 already had the content Print(result); initially. So, it is clear that it was deleted on source (yours) and untouched on destination (mine). The conflict can be automatically resolved now; all we need to do is keep the deleted line.

This is the magic of 3-way merge. Most of the actual file conflicts will be automatically resolved using very simple logic: If you modified one line and I didn’t touch it, then yours is the one to keep.

The good thing here is that 3-way merge can solve many conflicts in a single file. Let’s see another example: This time, both lines 30 and 70 enter the scenario. First, one user modified line 30 while at the same time the other deleted line 70. Still, a 3-way merge tool can merge the file without manual intervention. The figure shows the base and the two contributors and then the result.

As you can see, applying very simple logic, it is possible to conclude that we must keep line 30 from the destination (marked in green in the result to highlight the chosen contributor). Line 70 must be kept from source like in the previous example. The result marks the deleted line in blue to reflect the contributor it is coming from.

Now imagine you have to merge from a branch where 300 files were changed while the same 300 files were also modified in the destination branch. Thanks to a 3-way merge, the merge most likely won’t require manual intervention because the very simple logic behind the text merge algorithm will be able to solve all conflicts automatically.

The actual logic to automatically solve conflicts can be summarized as follows:

Now, what if two developers modify the same line divergently in two different ways? Check the following figure where the two developers modified the for loop differently. It was from 1 to 20 initially, but then one decided to set it as 1 to 15 while the other changed to 1 to 25.

The merge tool doesn’t know what to do in this situation and will ask for human intervention. This is what we call a manual conflict.

As you can see, the key is to have a base or common ancestor to run 3-way merges. Otherwise, all merges would be manual.

All the links and arrows in every single diagram in the book have a key purpose: Provide the merge algorithm with the information it needs to find the common ancestor between any two nodes in the graph. This is the way to find the right base for every single file involved in a merge.

Are you familiar with any of these tools? Kdiff3, p4merge, BeyondCompare, WinMerge? They are all merge tools. Plastic comes with its own 3-way merge tool, called Xmerge, and we think it is much better than all the others. But my point here is not to sell you the merits of our tool, but to explain what they all have in common.

As you saw in the examples above, when you do a 3-way merge, you need to handle the source (the changes you are merging from), the destination (your copy), and the base (how the file was before any of the changes). The drawings in the previous sections showed a very typical layout. It is, in fact, the one we use:

An alternative one, used by many merge tools, unified the result/destination as follows:

I prefer the first option (I mean, that’s the one we use after all), but you need to keep in mind that there are two main alternatives. Once you know that, no merge tool will be difficult for you. They might have more or fewer buttons, a simpler or more cluttered UI, but once you know the key elements in any tool, it is just a matter of just finding what you are looking for.

To find the common ancestor of two given changesets, the Plastic server takes the changeset graph (the same structure you see in the Branch Explorer) and walks back the tree until the paths meet.

As usual, it is much easier with an example:

The algorithm starts two walks from D and S, and their paths collide first on B, which is the base or common ancestor.

The base for the merge is the nearest common ancestor because, in the example, the changeset 131 is also a valid common ancestor, but it is not the closest to the two contributors.

Of course, life is not always that simple. Finding the common ancestor is not that obvious as in the previous example. Check the following figure:

Note that the case is not super difficult either, but as you can see, you have to walk through several branches before finding the changeset marked as B. It is a pretty obvious case, though, exactly like the first one but with a longer tree walk to reach the base.

What makes merge tracking slightly more complicated is that every new merge is affected by the previous merge history (merge history is equivalent to merge tracking, so you can also use it interchangeably).

Try to find the common ancestor in the following figure: (don’t go to the solution until you make a guess 😉)

I want to merge branch bug3001 into task701, so Plastic will walk back from S and D and consider the merge arrows as traversable paths. So, the result will be as follows:

Yes, changeset 134 is the actual common ancestor, thanks to the merge link. Otherwise, it would have been 131.

All these cases I have described so far are pretty simple, and this is how most typical merges will be most of the time. However, there will be situations when more than one common ancestor will be found at the same distance. And then, recursive merges will enter the scene. But, before going any deeper into merge tracking, let’s discover what happens once the right common ancestor is found.

We’ve arrived at the core of the merge process! You are about to become a master in the internals of merge, and understand how Plastic merges branches. Armed with this understanding, you will no longer have trouble understanding even the more complex merges.

Of course, we are going to start with a branching diagram:

We want to merge bug3001 into main. The common ancestor is already located, and the diagram is decorated with the actual changes.

To calculate the merge, Plastic doesn’t load all the individual diffs made in 3 and 4 and then destination and source. It doesn’t accumulate the changes to find the solution. (This is not a bad alternative and some systems like Darcs do something similar, but we do it differently).

What Plastic does is load the tree at base, source, and destination and then calculate the diffs between source and base, then destination and base and work based on that.

Let’s go step-by-step. First, I’m going to create the tree for changeset 1:

The numbers are the revision ids. Every object in Plastic has an object id, which is an int (4 bytes). It is also identified by a GUID that makes it unique even across replicated repos. I will use the revision id (revid for short) but the example would also work with GUIDs.

It is also important to note that every file or directory also has an item identifier (item id). This means bar.h might have many different revisions, but they all belong to a given item id.

Now, let’s go to changeset number 2, the base, where we only changed /src/bar.c. This is what its tree looks like:

I marked in green the nodes that were modified from the previous changeset 1. As you can see, there is a new revision for bar.c, then for its containing directory src/ and then the root /. The rest of the tree remains unchanged.

For the sake of clarity, I drew all the trees in the example:

I think this graphic gives you a pretty good explanation of how Plastic works. Of course, it is very good to understand merge, which is our goal now.

As I said, to figure out the actual changes since the base Plastic doesn’t load all the trees, it just loads the base, source, and destination.

Once the trees are loaded, Plastic first calculates the diff between the base and destination trees. It will be something as follows:

The merge code walks the two trees as follows:

Now, the differences between base and source:

Here:

Now, the algorithm takes the two pairs of differences to find conflicts. It can mount a result tree like the following:

As you can see, it can already mount /inc/foo.h in the tree, and it knows it has to load /src/foo.c revid 10. But, there is a conflict in bar.c because concurrent changes happened on both contributors.

Let’s now put the three trees together:

To solve the conflict in bar.c, we’ll need to run a 3-way merge loading revid 7 as base, 13 as src, and 16 as dst.

And this is how everything comes together! Find the common ancestor to locate what changed in the two contributors and then locate the files' common ancestors in conflict 🙂.

I don’t think I’ve ever drawn so many graphics before in my life 😃!

If you went through all this and got it, now you can say you are an expert in merging. Congratulations!

Merge tracking happens at the changeset level, not the file level.

When Plastic tries to find the common ancestor of a merge, it does it on a tree-wide level.

I’m going to explain what this exactly means and the implications it has.

Consider the following scenario:

You only want to merge /src/bar.c to main, but skip the others because you are not interested in them for whatever reason. Plastic can’t do that but suppose for a moment that you manage to make it happen.

Now, you’ll have a resulting changeset where only bar.c was merged from bug3001 and a merge link set, as follows:

And finally, you want to get foo.c and foo.h into main. You try to repeat the merge but the merge link between source and destination clearly says the merge is impossible because there is a direct path between them (source is a parent of destination, so no merge can happen).

Do you see the problem now? That’s why when you merge a branch or changeset, you merge all or nothing because otherwise, we’d render the merge links useless.

The alternative to this is per-item (per file, per directory) merge tracking. Plastic and Git (and any modern version control) uses per-changeset, but SVN, Perforce, TFS, and the now-archaic Clearcase, use per-item merge tracking. We used per-item before version 4.0.

What are the cons of per-item?

Merge tracking can save tons of manual work by automating merges that otherwise would require manual intervention. We already saw it with some simple 3-way merges. Now, I’m going to walk you through a very interesting case where you’ll discover why having proper merge tracking makes a huge difference.

Let’s start with this scenario:

We want to merge fix-7.0 back to main, and there is a conflict on /src/bar.c.

Let’s take a look at the actual files and solve the conflict:

Look at the figure carefully:

Then, we continue working on our project, making a few more changes as follows:

And now, we want to merge bug3001 into task701. The common ancestor will be changeset 104 (the old source this time). Let’s zoom into the file contents to solve the possible conflicts:

What happened is that in task701, line 51 was modified to be 1 to 45. Meanwhile, in branch bug3001, line 70 was added with content Print(result++);.

The result of the merge can be solved automatically by a 3-way merge tool.

Well, you may say, this is pretty normal. The 2 branches don’t touch the same line, so why should it be manual?

Let’s take a look at how the merge would be if we ignore the merge link between 104 and 105:

Not only is the merge not automatic anymore, but you have two manual conflicts instead of one!

Take your time to look closely at the figure and understand why there are manual conflicts now. It will really pay off.

In this chapter, we always had two contributors (source and destination) and one common ancestor in all the examples. All the trouble was finding the ancestor, but the only difficulty was walking the tree a little bit further. Of course, the ancestor could be harder to find in a forest of branches, but it wouldn’t be that hard for an algorithm.

Now, what if there is more than one ancestor found at the same distance? Let’s see an example:

We call this a crisscross merge scenario. The case itself is forced because there is no good reason to merge from 11 to 16 when 15 already exists. But, as you can imagine, this could happen in a distributed scenario where the merge 11 to 16 happens on a repo where 11 is the head of main. In contrast, 12-15 happens on a different one where only 12 was pushed at the time.

Anyway, the scenario in the figure is the shortest one driving to a merge with multiple ancestors, but the same can happen with several changesets and branches between the "crossed" merges, as we’ll see later.

As you can see in the figure, both changesets 11 and 12 are valid nearest common ancestors because they are both at the same distance from 16 and 15. So which one should we choose?

The answer is not choosing one because a random choice will lead to incorrect merges, but running a recursive merge where a new unique virtual common ancestor is calculated.

The figure shows the new virtual ancestor that Plastic creates by merging changesets 11 and 12. Then its resulting tree is used as ancestor of 15 and 16 to resolve the merge.

Plastic does all this operation under the hood when it finds multiple ancestors, so normally you are not even aware of what is going on and the merge simply happens correctly.

There is some effect you can notice, though. Suppose a file foo.c is modified by 11 and 12, so it needs to be merged, and there are manual conflicts. Plastic will ask you to solve those conflicts to use the resulting file as the basis for the next merge. Then, if the file was again modified in 15 and 16, you could have to solve a manual conflict again. The effect is that you’ll see the merge tool show up twice for the same file. Now you understand why it can happen 🙂.

Once the basics of recursive merge are clear, let’s jump into a more complex case, where more than two ancestors are found at the same distance.

Take a look at the following figure:

As you see, now we have three ancestors at the same distance: 11, 12, and 13. So how does Plastic solve this issue?

It will chain virtual ancestors, merging them in pairs recursively (that’s where the name comes from) until only one is left. This final virtual ancestor is then used to solve the original merge.

The next figure explains ancestors V1 and V2 are created and V2 is used as the common ancestor for 16 and 17.

Now, you have a perfect understanding of how Plastic solves even complex merge situations. Admittedly, you won’t be facing recursive merges very often. Still, it is good to have a proper understanding of what is going on when they happen.

A last note: Do you remember the entire explanation in "Merging trees"? In essence, Plastic is merging three trees, and that’s why a solution like the one described here is possible. The "tree merge" code takes three trees and then produces a result. All we have to do is give it a different tree, in this case, a virtual one, so it can operate and create a meaningful result.

You might be wondering: "Why all this trouble and complexity instead of just picking one of the possible ancestors to solve the merge"? Ok, let’s see why with an example.

In the following scenario, there are two possible common ancestors at the same distance of contributors 6 and 7. As you can see, this scenario is not a crisscross. Instead, it is a more realistic one, but multiple ancestors at the same distance are found again.

First, let’s explore what happens if we randomly choose changeset 4 as common ancestor and perform the merge. By the way, I’m using the following image to introduce the file’s actual contents in the conflict in the example.

While delving into 3-way merge, we learned this would be a merge with manual conflict resolution since line 40 was modified differently by the two contributors. So, choosing 4 as a common ancestor will end up in a manual conflict.

Let’s now check what happens if we choose 3:

This time, the conflict would be automatic, and the result would be keeping the Print("See you"); sentence.

Which one is better? Without recursive merge, you have to rely on luck. Depending on what the algorithm does, it could pick 3 or 4. Then you can end up with conflicts or even with a wrong automatic resolution. I mean "luck" because there is no heuristic to actually make a better choice.

Now, let’s explore what recursive merge would do; it will merge ancestors 3 and 4 to create a new virtual one as the common ancestor for changesets 6 and 7. Ok, let’s go ahead:

As you can see, the virtual merge will be automatic; it will choose Print("Hello"); in line 40. No conflicts.

Let’s now use this result as the virtual ancestor for 6 and 7:

Now, it is clear that the merge is automatic again, and the final value for line 40 is Print("See you");.

Recursive merge can solve this conflict automatically in a consistent way, and the randomness of choosing one selector over the other is gone.

How important is this kind of resolution? Well, consider a real-world scenario with a few hundred files involved. Suppose your branch history leads to a multiple common ancestor scenario. You could potentially be saving tons of manual conflict resolutions by taking advantage of recursive merge 🙂.

Directories are first-class citizens in Plastic. This means they are versioned in the same way files are. Plastic can have empty directories because they are just not containers of files but entities on their own.

Here is a simple tree of a given changeset:

What does the actual data of revision 10 look like? Revision 10 belongs to a file named foo.c. Let’s see what its associated data looks like:

The image shows the contents of the associated blob plus its metadata. As you can see, this small fragment of text is not compressed, and since it is tiny, it is the segment zero (we split large files into compressed segments of 4MB each).

Now, what is the data of revision 11? This revision is a directory. So, what’s the actual data of a directory? You are about to see one of the key data structures that supports the Plastic merge power 😉.

Look at the image carefully. The data of a directory are the names (plus some metadata) of the files it contains. This has a profound implication on how Plastic works and merges directories. Files don’t have a name, they are not even "aware" of their name because directories hold the names. This is going to be key in how directory merges are handled.

Suppose we want to rename src/foo.c into src/boo.c. What will the resulting tree look like?

Exactly. When you rename boo.c, the revision of boo.c doesn’t change. What changes is its container src/. So, a new revision of src/ is created but not a new one of boo.c. src/ doesn’t know its name either, its parent (/ in this case) holds its name.

Let’s now see what happens if we decide to move /src/boo.c to /inc/boo.c:

Again, boo.c won’t change, only its source and destination directories will.

An essential part of merge calculation is diffing trees, as we saw in the previous sections. Here, I’d like to explain how moves are calculated between any two different trees and how the metadata helps find these moves.

Suppose we have a tree similar to the previous section, and we want to calculate the diffs between changesets 12 and 28.

The operations that happened in the changes between the two were a move of /src/boo.c to /inc/zoo.c and then a change in the new /inc/zoo.c. I describe the two operations in the figure for the sake of clarity. This doesn’t mean both happened in changeset 28. They could have happened somewhere in between the two.

When Plastic takes two trees to diff, it starts walking the destination and finds matches in the source. Destination is 28 and source is 12.

The actual result is as follows:

Basically, the diff finds that:

The next phase in the diff is trying to pair added and deleted to find moves. In this case, the itemid matches, clearly a move between /src/boo.c and /inc/zoo.c.

Pay attention to the following: The revision ids don’t match. Since zoo.c was modified between the changeset 12 and 28, its revid is now 86 while it was revid 10 in the origin src/boo.c. So revid alone is not enough to match moves. That’s why Plastic uses an itemid to clearly identify elements.

The concept of the itemid is crucial in Plastic because it is key for move tracking. It is certainly costly to maintain, but the results pay off because merging moved files and directories becomes straightforward.

For instance, the same logic applies if, instead of moving a file, we move a directory to a different location:

As you can see in the figure, the diff finds /inc as deleted and then /src/inc as added but it can pair them as a move since they have the same itemid. I didn’t draw the children of the moved directory in the diff tree because there are no changes and the algorithm prunes the walk when there are no more differences. In this case, the diff is even simpler because the revids also match, but of course, it would work perfectly too if the revision id of the directory changed.

All the hassle of dealing with itemids pays off to perfect merging moved files.

Moving files and directories are essential for refactoring. Not only that, in some programming languages, directories are directly associated with namespaces, and a change in a namespace can lead to tons of directory renames. It is also possible that restructuring a project is good for keeping the code clean, readable, searchable, and ultimately findable.

One approach is to postpone those changes until "we find a moment of calm with fewer concurrent changes". This never happens. The reality is that if you can’t refactor your code and directory structure while you continue with normal development, you never will. It is the same as postponing code reviews or any other key practices. You better have the practices and tools to do them daily, or you will postpone them indefinitely.

I hope I explained clearly how important it is to freely and safely move files around. That being said, let’s see a couple of scenarios in action.

The example in the figure is the canonical "merge a moved file" sample:

The merge result must keep the rename /zoo.c and combine the changes made by the two developers.

In the previous sections, we saw how Plastic diffs the trees to find actual conflicts and a valid common ancestor to merge them. As you now know, the actual names are not relevant for the algorithm since it can identify files and directories by itemid. That’s why this case is straightforward in Plastic; it can safely merge zoo.c because it knows it is the same item regardless of the names it has on both contributors.

Let’s now complicate the scenario a little bit by involving directories.

Let’s examine this second scenario:

And again, as convoluted as the scenario might sound, Plastic only finds a conflict in /core/render/bar.c because it knows the two contributors modified the item. So, it will run a 3-way merge to resolve possible conflicts while keeping the directory and file moves in the result.

Change/delete conflicts happen when someone modifies a file while someone else deletes it in parallel.

Since the new change can be significant, Plastic warns of the situation during the merge so the developer can decide whether the change should be preserved or discarded.

The simplest scenario is as follows:

Plastic warns of the case during the merge and lets you recover the change discarding the deletion if needed.

A more complex case involves a deeper directory structure and not-so-direct deletions. Look at the following figure:

Plastic will detect a conflict and ask you what to do.

Of course, this type of conflict will only be detected if actual elements are in danger of being lost. I mean, consider the following case where we have a directory tree as follows:

Then, the developer working on the branch named task does the following:

In this case, there won’t be a merge conflict since boo.c was "saved" during the moves done previous to the deletion of /src/render, so the changes made in main will be applied cleanly.

Add/move conflicts happen when you add a file or directory, and concurrently someone moves an existing file or directory to the exact location where you added the file. Again, Plastic will help you solve the conflict.

The example in the figure matches what I described before:

Here itemids come to the rescue. Plastic is not fooled by the fact that the files have the same name. It knows they are different files because they have different itemids. So, Plastic will ask you to keep one or the other or to keep both by renaming one of them.

I won’t describe a more complex case with directories because it will be the same as we just saw. Finally, a combination of files and directories where names collide is also possible (suppose the added bar.c was a directory), but the resolution will be the same as described above.

Move/delete conflicts happen when you move a file or directory to a different location, and concurrently someone else deletes the destination directory. If not handled correctly, a merge of the concurrent changes could lead to valuable data loss.

Let’s see an example:

When you merge his branch into yours, Plastic will warn you about the move/delete conflict and will let you go ahead and remove the file or keep the delete.

This type of conflict makes more sense when the scenario involves directories. Look at the following figure:

When the branches get merged, Plastic will detect the move/delete conflict, warn you about the situation and let you decide whether you want to keep the delete and lose the moved file, or undo the deletion.

Suppose you undo the delete of /core/utils during the merge. In that case, chances are you will "restore" many other files and directories inside utils. But, they will stay in your workspace after the merge, and you will have the chance to delete again what you don’t need, but also to preserve sum.c which is what you wanted.

Evil twin conflicts happen when two different items collide in the exact location. I don’t remember where the dramatic name of "evil" came from. Not sure if we took it from Clearcase, if we found it somewhere else or if we came up with it.

Anyway, the conflict is easy to understand with an example:

The merge will detect an evil twin conflict because the directory can’t have two items with the same name.

Plastic proposes these resolution options:

A moved evil twin conflict happens when you move two files or directories to the same location in different branches, and you try to merge them. The directory can’t contain two items with the same name, and a conflict is raised.

Check the following example:

This scenario usually happens when similar functionality exists in two different places (like the case here).

In this type of conflict, Plastic provides three options:

As with regular evil twins, it could be interesting to do a 2-way merge if we know the files are the same although created separately. If you think this is something interesting, please let us know.

This is my favorite merge conflict and the one I often use in demos because I think it demonstrates the strength of the Plastic merge engine.

As usual, let me explain it with a graphic:

Plastic detects the conflict on merge and asks which name you want to keep.

Things can get more complex if we involve directories together with concurrent changes on a file:

Plastic will first ask you to decide where to put the gl directory. For example, do you want to rename it and keep it under engine? Or move it to /engine and rename it to ogl?

And, once the directory conflict is solved, Plastic will ask you to merge the mesh.cpp file.

Why do I like this type of conflict so much? Well, because it turns a complex (although obvious) scenario into something fully manageable while other version control systems (Git among them) will turn this into a complete nightmare. What Git does is duplicate the directory and all its contents, which makes unifying mesh.cpp extremely complex and unlikely to happen, which usually leads to losing changes.

Cycle moves are not common, but they can happen if you move one directory inside another (src into code/src). In the meantime, somebody else performs the opposite operation (code into src/code).

When Plastic detects this case, it asks the user which move to keep because it can’t keep both.

Internally, it could create a infinite loop where one is a child of the other recursively (not a good thing).

Some version controls duplicate the trees, so you end up with /src/code and /code/src and all the files duplicated: It doesn’t look like a good conflict resolution strategy 😉.

When you solve a complex merge in Plastic, you’ll always have to solve directory conflicts (structure conflicts if you prefer) before file conflicts.

This is because you need to have a proper structure before "plugging" the files into it. So, suppose the directory structure needs you to solve whether you want to keep the rename from bar.c to foo.c or boo.c. In that case, you first have to solve that before merging the file’s content.

Both the command line and the GUIs follow this order to solve conflicts.

The following screenshot taken on Windows (macOS and Linux follow a similar pattern) shows a divergent move directory conflict followed by a file conflict on the same file; you first have to solve the directory conflict and then merge the file.

For each type of directory conflict, choose whether the source or the destination contributor should be automatically selected to resolve the conflict. See the following example:

The above merge operation from changeset 2634 resolves the eviltwin and changedelete conflicts by keeping the source (-src) contributor in both cases.

Here is the complete list of conflict types the merge command supports:

The all value overrides the other options. In the following example, eviltwin-dst will be ignored:

Subtractive merge is a powerful merge operation that allows you to remove a given change while preserving the rest of the changes made later.

Consider the following scenario:

You need to get rid of the changes made in changeset 92, but the others are perfectly fine.

Deleting 92 is not an option since, as we saw in "Delete empty branches only" it is only possible to delete 92 if you delete 93, 94, and 95.

Subtractive merge comes to the rescue here as follows:

By "subtracting" 92, you create a new changeset 96 that only removes the changes in 92, and keeps the ones done later.

A simple rule of thumb: Don’t abuse subtractive merge. If you find many red lines in your repo (that’s how we render subtractive), be very careful. Subtractive is a tool to handle exceptions, not something to use daily.

This is because subtractive merge links are informative: They help you understand what happened, but they are not actively considered for merge tracking. And, you already know why merge tracking is so important to avoid issues. So, use it wisely.

Consider the following scenario: A couple of new branches were merged to create a new version, but then after some more intensive testing or during internal deployment, you detect the release is broken and figure out that branch bug088 is the one that introduced the bug:

How can you proceed here?

My recommendation, under normal circumstances, would be to implement a new task to fix the issue and solve the broken build:

I like this solution because it doesn’t introduce an exception case. It simply sticks to the same "task branch" and regular DevOps cycle. There is no manual intervention needed, just a new task branch that will be tested and merged by the CI system or a Plastic mergebot, and everything will proceed as usual even if we are fixing an unusual emergency provoked by a broken build.

Well, this is the "Subtractive merge" section so you most likely expect me to use a subtractive merge to solve the scenario, right?

Here we go:

The other alternative is to take advantage of subtractive merge to remove the changes made on bug088 (we are subtracting the changeset result of its merge, so all its changes will be gone) while preserving the later bug089 which was perfectly fine.

Subtractive is powerful, but it is not a beginner’s tool.

In the purest "task branches" philosophy, you would abandon bug088 and create a new task to fix the new issue. But, sometimes that’s not doable because it is better to fix whatever happened to bug088 and merge again.

And it is the "merge again" that is worth a section since it is far from straightforward.

Let’s continue with bug088 and fix the issue we introduced there by doing a new checkin on that branch. Then, it would be possible to merge it back, right?

Wrong.

Now, you have an excellent opportunity to practice all the newly gained expertise in merge tracking. What happens when you merge 160 into 157?

Well, first 143 will be the common ancestor, and only the changes made in 160 will be merged. It means you are not really introducing all the changes done in bug088, but only the new changes in 160.

This is because the subtractive merge link is informative and not part of the merge tracking calculation. So it is completely ignored.

What is the right way to do it?

Pay attention to the following figure:

To "reintegrate" a previously subtracted branch with newer changes, you have to:

The procedure is not hard, but you must know how to use this feature properly. I hope the explanation helped you master subtractive.

When you cherry pick a changeset, it means you want to apply the changes made in this changeset to a different branch. It is like applying a patch just taking the changes made in the changeset.

The following figure explains the case: By cherry picking changeset 15 to branch maintenance-3.0, you are just applying the changes made in 15 to the branch.

What would happen if you merged from changeset 15 instead? This is very important to differentiate because users often get confused about the difference between cherry pick and merge. Look at the following figure:

A merge from changeset 15 would apply the changes made in 11, 13, and 15, not just the ones in 15 as cherry pick does. That’s the key difference between cherry pick and merge.

We can also apply a cherry pick of a whole branch instead of just a changeset.

Consider a scenario like the following: we want to cherry pick main/task12 to main instead of merging it.

The cherry pick will take the changes made on changesets 6, 8, and 9 into main, while the merge would also bring the changes made in 3 and 5 in the parent branch main/task10.

To put it in more formal terms (which I bet some of you will appreciate), the merge in the figure considers the (5, 9] interval; changesets 6, 8, and 9 will be "cherry picked".

Branch cherry picking can be very useful under certain circumstances when you want to pick a few changes instead of the whole history.

Branch cherry pick is indeed a subset of the interval merge, where the topology of the branch sets the interval.

The figure shows an interval merge of (5, 9] which means the changes made in 6, 8, and 9 will be cherry picked to main.

What happens if two team members modify an unmergeable file concurrently? Suppose you have a .jpg file and two artists made a change to it concurrently.

Plastic comes with a binary merge tool that is invoked only if a file is detected as binary (Plastic bases its decision on the file extension for known types, the filetypes.conf config file, and an algorithm that reads the first bytes of the file and figures out if it is binary or text. There is also a way to configure a mergetool based on the file extension). So, in this case, since ironman.jpg is detected as a binary, it will invoke the binary merge tool, which will only let the user choose which contributor to keep and which one to discard.

Since it is a jpg file, the binary merge tool can show a preview to help you with the decision as follows:

While it shows you the common ancestor and the two contributors, it cannot actually merge the pictures to produce a new one, and one of the changes will be discarded.

If the file in conflict was not an image, the binary merge tool won’t be able to create a preview (unless you plug a preview generator, but that’s a different story). What you would see in this case would be something as follows:

Anyway, what is clear from this example is that while there is a way-out during merge, it is not a good idea to discard work trying to work in parallel in files that can’t be merged.

Plastic can be configured to lock certain files and prevent concurrent changes (check this for more info).

I will explain how it works and what the team needs to do to take advantage of this feature.

First, the sysadmin needs to configure locking on the server for certain file extensions. In my case, I’m going to consider that all .jpg files are locked on checkout.

Then, let’s consider the following starting point:

Violeta and Carlos are working on the same project. Violeta decides to checkout the file ironman.jpg. In the figure, the checkout is done using the command line, but of course, the same operation can be done from a GUI.

Checkout is an operation to tell Plastic "I’m going to modify this file". Usually, it is not needed because you can modify the file without telling Plastic about it, and it will find out later.

But, checkout works differently for files configured to be locked. When you checkout ironman.jpg, since it is configured to be locked, a new entry is created in the server lock list, meaning that ironman.jpg can’t be modified concurrently.

In our example, Violeta correctly checkedout ironman.jpg, and the file was added to the server lock list. I just added the file’s name to the list in the figure above, but in reality, info about the itemid is also added to identify the locked files properly.

Suppose that now Carlos tries to checkout ironman.jpg too:

The checkout operation fails because Violeta already locks the file in a different workspace. This is how we can prevent unmergeable files from being wrongly modified concurrently.

Later, Violeta finishes her work and checkins the changes. A new changeset 13 is created on main, and the entry in the lock list is now cleared.

Now, Carlos is free to lock ironman.jpg to perform changes, correct? Not quite yet:

Carlos is still on changeset 12 in his workspace, so he can’t lock the file because, while there aren’t locks for ironman.jpg held at this point, his revision is not the latest.

Plastic ensures you can only lock the latest version of a file otherwise concurrent modification wouldn’t be prevented.

Then, Carlos decides to update his workspace to changeset 13, and he is finally able to lock the file.

This way the changes in the jpg file will be serialized since nobody will be able to make concurrent modifications.

The Plastic locking system only works well if you work on a single branch. This is because we didn’t implement the ability to coordinate locks among branches. You can easily force the situation where you held a lock on a branch, so nobody else can modify the file, then checkin. Of course, at this point, someone else could lock the file on a different branch. But this won’t prevent concurrent modification.

In the figure, someone created changeset 13 after locking ironman.jpg on main.

Later someone creates a branch starting from changeset 11 and locks the file (checkout). Their version of ironman.jpg is the latest on their branch, so they can perform the lock, but obviously we are not preventing merges since this is in fact a concurrent modification.

So, keep in mind that locking is very useful to deal with unmergeable files, but you need to restrict its use to single branch operation.

Let’s continue with the same sample repo, but suppose my workspace was located on changeset 3 instead, and we decide to run an update.

The structure of the trees of the changesets 3 and 5 are as follows:

So, when you decide to run an update, Plastic will realize it is in changeset 3, and it has to jump to changeset 5. It knows it has to download a new version for bar.c replacing the current one. At the same time, it downloads new data to update /src/bar.c, it will update plastic.wktree accordingly.

Update moves the workspace to its latest configuration. So, suppose if your workspace is set to work on main, and you run an update, Plastic will locate whatever the branch’s head is and try to jump to it, updating both data and metadata in the workspace.

Instead of going to "latest", a switch is an update that jumps to a different configuration, typically a different branch or changeset.

Let’s start from where we left in the previous section:

And let’s run a switch to main/task2001:

Plastic will check the trees of the changesets 5 and 6:

And then figure out what needs to be downloaded to convert your working copy into changeset 6.

In this case, a new file inc/foo.h will be downloaded from the repo (it didn’t exist on the workspace before), and the contents of both src/bar.c and src/foo.c will be updated to different revisions.

It is possible to update a workspace to a different repository. In Plastic, workspaces are not tied to a given repo: You can reuse a workspace to work with a different repository at any time.

The update and switch operations will try to reuse as much data as possible to reduce the traffic with the repo (which might be a distant server). I mean, even if the revids don’t match, Plastic will try to reuse files based on hashes. So, if you switch your workspace from main/task2001@quake@skull:8084 to main/task2001@quakereplica@localhost:8084 where quake and quakereplica are replicas of the same repo, Plastic will try to reuse all the files on disk, and if changeset 6 exists on both, no new file data will be downloaded.

The only practical reason why you wouldn’t switch a workspace, though, is size. I mean, suppose your workspace is 3GB, and you have local files that add another 2.5GB. If this workspace points to quake@skull:8084, and you plan to continue working on this repo, there is no good reason to switch to doom@skull:8084 and have to download another 5GB and discard the 2.5GB of local private files.

cloaked.conf defines the paths to cloak in the workspace. The file is defined as a set of user-defined regular expressions. The files that match any of the rules will be cloaked.

Cloaked works in two different ways:

You can read the details of how to setup cloaked.conf and check the reference guide to learn how to configure it from the GUI.

By default, Plastic sets all files on the workspace as writable. It is the default behavior since most developers just want to edit a file and save it without dealing with overriding the read-only attribute.

But, sometimes, it is useful to set files as read-only so that users see an operating system warning asking them to override the read-only setting when they hit save in their applications. This is commonly used by team members working on assets that need to be locked, as a reminder to checkout the files before saving them.

Plastic provides a global setting to decide whether an update should set files as writable (the default) or read-only.

The writable.conf and readonly.conf config files are useful to create exceptions to the global setting.

In a default "everything is writable" configuration, readonly.conf will define which extensions or file patterns must be set as read-only on the workspace.

And writable.conf does exactly the opposite.

You can read the details of how to setup writable.conf and readonly.conf and check the reference guide to learn how to configure this setting from the GUI.

Admittedly, this is one of my least favorite settings. We only added it to ease the transition of teams to Plastic from their previous version control.

The file eolconversion.conf defines which files need to convert end of lines during the update and reverted back to their original during checkin.

I’m not a fan of this option because otherwise, Plastic never touches the file content during update or checkin, preserving what the developer created. Converting EOLs introduces a performance impact and breaking the "we preserve what you wrote" rule.

Nevertheless, some teams prefer to see Windows-style EOLs on Windows and Linux-style EOLs on Linux and macOS. Nowadays, almost all editors I’m aware of are capable of handling EOLs transparently. Still, some teams and some tools prefer that the version control normalizes the end of lines.

Learn how to specify eolconversion.conf.

Each time you run a cm status on the command line or go to "pending changes" or "checkin changes" in the GUIs (we use the two names interchangeably), Plastic walks the workspace to find any changes.

It groups the changes in the following sections:

Check the following figure, based on the scenario above where the workspace points to changeset 6:

I didn’t cover how moves and renames are detected since they have their own section.

Now, you have a pretty good understanding of how changes are detected in a workspace.

Any file or directory you add to the workspace will be considered private. Private means it is just local to the workspace and won’t be checked in, and changes won’t be tracked.

Each time you ask Plastic to find changes, the local files will show up marked as private.

All these privates are suggested by the GUIs to be added to source control.

Very often, that’s exactly what you want to achieve because you just added a new file and want to add it to version control.

But there are many files, like local configs or intermediate build files (obj, bin, etc) that you want to ignore, so Plastic doesn’t show them repeatedly.

All you have to do is to add them to ignore.conf and Plastic will, well, ignore them 🙂.

Although adding files to the ignore.conf list is very easy from GUIs or simply editing the file. You can read the details of how to setup ignore.conf.

In the chapter about merges, you already saw how good Plastic deals with moved files and directories and how important it is to help refactoring the codebase.

But, before a move is checked in to the repo, first, it has to be detected on the workspace. This is what I’m going to explain now.

Let’s start again with changeset 6 in the examples above and consider the following local changes:

The file inc/bar.h is no longer on disk, and a new one inc/ren.h appeared as new.

If move detection is enabled (by default, but it can be disabled), Plastic tries to match added/deleted pairs to try to find moves.

The following figure gives an overview of the algorithm:

The move/rename detection also works for directories. Let’s see it with an example: Suppose we rename the inc/ directory to incl/. Then, the change detection algorithm will see is something as follows:

It finds that inc/ and its children are no longer on disk, and a new tree incl/ has been added.

Then, the added/deleted pairs will try to be matched. In this case, the algorithm finds that inc/foo.h and incl/foo.h have the same size and content, so they must be the same file. And the same happens for ren.h.

But instead of showing a pair of moves, the Plastic algorithm goes one step further: It tries to match inc/ and incl/. Since its contents are the same, it detects a rename of inc to incl and cleans up the individual file moves.

Suppose the files were modified, deleted, or moved after the directory rename. In that case, Plastic will apply a similarity percentage to the inc/ directory structure. Suppose the structures are similar up to a given %. In that case, the directory will be considered as a move/rename instead of added/deleted.

We are very proud of move detection, and we gave this feature a big name: "Transparent scm" or "transparent version control" because you can simply work in your workspace, make any changes, and trust that Plastic will detect what you have done.

We use several extra techniques to detect moves, but I’ll cover that in its own section.

Just like you sometimes need to ignore private files with ignore.conf, there are scenarios where you don’t want Plastic to detect changes in specific files because you don’t want to accidentally include them in the next checkin. This can happen if you need to adjust config files for a debug session or alter some source code for whatever reason, but you don’t want to checkin those temporary changes.

hidden_changes.conf helps achieve that. You can read the details of how to setup hidden_changes.conf and check the reference guide to learn how to configure it from the GUI.

Suppose you want to heavily refactor your project. You can expect Plastic to correctly guess what you did after the fact based on similarity algorithms. What is the alternative?

Let’s start again from the tree in changeset 6:

And let’s perform the following actions:

Now, let’s check the metadata of the workspace:

As you can see in the figure:

Note that I marked the tree with a "CO" instead of a number. CO stands for checkout. plastic.changes contains what we call "the checkout tree", which is a tree that contains the local changes controlled by Plastic and ready to be checked in.

There is something important to note: When you tell Plastic to move, add, or delete, Plastic doesn’t have to guess these operations anymore. Unlike what was described in the "Finding changes" section, these are controlled changes; Plastic knows about them because you told it you did them, so there is no guessing or heuristics involved.

One of this alternative’s key advantages is to handle moves and renames; there is no guessing involved, so you can perform extremely complex refactors with "moves" and Plastic will track exactly what you did.

After performing these changes, if you go to the Branch Explorer, you’ll see something as follows:

See how there is a new "checkout changeset", and the home icon is on it, meaning your workspace is not synced anymore to changeset 6, but 6 and some controlled changes on your workspace.

If you make "not-controlled" changes (make the moves and deletes and let Plastic detect them automatically), then there won’t be a checkout changeset in your Branch Explorer.

You can tell Plastic you are going to modify a file by using the checkout action both from GUIs (right-click a file from "Workspace Explorer" and select "checkout") and command line (cm co command).

Let’s continue with the example above. You can perform:

And Plastic will mark the file as checkedout both in plastic.wktree and plastic.changes.

This is what we strictly call a checkout, although, as we saw in the previous section, we extend the name checkout for controlled deletes, moved, and added files.

Very simply, to control changes:

Usually, you will use a combination of controlled and not controlled changes; most likely you’ll simply modify files, but sometimes you’ll delete or move files from the Plastic GUI. Plastic is able to deal with both types of changes simultaneously.

Walking the full workspace tree to look for changes means reading the entries of each of the directories and comparing size, name, and last write times with the workspace metadata (plastic.wktree).

Doing this walk in a naïve way proved not to be fast enough, so:

In Windows, we can use the USN Journal on NTFS volumes to do precise move tracking. Linux and macOS don’t provide an equivalent feature, which is one reason we were reluctant to enable it in Windows. This means there is a way to precisely detect moves and renames, detected by the file system itself. The only downside is that many editors don’t really rename but create a new file with the new name and then delete the old one.

Suppose you want to modify changeset 5 with changes coming from 6 without using merges. In that case, you can shelve src/foo.c and apply those changes to the new configuration and checkin.

You might think, this will "merge the changes", not override them. Yes, that’s true. Suppose you want to override src/foo in changeset 5 with your changes made while working on a different branch. In that case, you have to manually copy the file, switch, then overwrite. I know this is not ideal, but what you are doing is not a recommended practice. It should be a rare case that you shouldn’t perform often, and that’s why we think we shouldn’t make it easier 🙂.

After you create your partial workspace (typically using Gluon, although you can also create a regular workspace from CLI and then run cm partial configure), the next step is to configure it. Configuration tells Plastic what files you want to download and track, which means the workspace will behave differently from a full one.

I will start with the example we were following in the chapter, and show the trees on the repo and how everything is mapped to the workspace.

The figure shows how we started working on our partial workspace when the only changeset that existed on the repo was changeset 1. Then, we configure the partial workspace to download only src/foo.c, and the workspace metadata will reflect that; it just loads foo.c revid 03. Note how I deliberately omit revision numbers for directories.

Also, note how the src directory is "partially checked" during configuration. This means not all its children are loaded, and it will impact how the update operation works in partial workspaces. More on that later.

Then, after a while, the repo evolves to changeset 2 (someone else did a checkin). Then we decide to configure bar.c as follows:

Note how the src directory is now fully checked because all its children are fully checked. Our local metadata now loads bar.c revid 07 and foo.c revid 03, so we could consider our workspace is synced to changeset 2 although that won’t last.

The repo continued evolving, and it is now at cset 5. And then, we decide to update just one of the files we have: we only update bar.c, but we don’t download the new changes of foo.c.

At this point, we couldn’t draw a "house icon" on any changeset on the Branch Explorer because the workspace is not "at any given changesest" anymore. So, what we have in the workspace is a mix of things on the repo, but they are not together in the same way they are in the workspace at any changeset.

Then, the repo continues evolving, and it reaches changeset 7. At this point, we run a full update, and the new src/new.c will be downloaded, together with the new version of foo.c that we refused to download before. The file new.c is only downloaded during the workspace update because the src directory is "fully checked". If it wasn’t fully checked, new files wouldn’t be downloaded during the update. At this point, we could say our workspace is again in sync with changeset 7 although this is not a common scenario with partial workspaces, which tend to stay "not synced to a given cset" for a long time.

Changeset 8 brings new changes for foo.c and new.c, but we only decide to update new.c. Our workspace isn’t in sync with a changeset anymore.

Then, changeset 9 introduces even more modifications (made again by someone else working on the repo), but at this point, we don’t make any updates. But, we modified bar.c locally.

Now, with our workspace not in sync with any changeset, we’ll try to checkin bar.c. The partial workspace can do that without requiring a previous update to be "in sync with head".

The following figure shows what happens after you checkin src/bar.c from the partial workspace. A new changeset is created in the repo (changeset 9 this time), but the local metadata is only updated for bar.c, while the rest stays how it was.

We created a new changeset on the branch, while our workspace doesn’t have the same file versions than the new head changeset we just created. This flexibility is the key benefit of partial workspaces.

Of course, don’t try to perform branch merges on partial workspaces. It’s not possible to merge to the current workspace status because it doesn’t resemble any changeset configuration, and hence merge tracking wouldn’t work.

Other than that restriction, partial workspaces are very useful for per-file version control workflows.

During the last few sections, I introduced the meaning of partially checked compared to fully checked directories in partial workspaces configurations, but I will explain them here in detail.

Consider we configure our workspace in the following way:

Note that since I didn’t select all the children of src/, it is partially checked.

Then, after a while, the repo evolved, and new changes to bar.c, foo.c appeared, and also a new file under src. Will the new changes be downloaded if I update the workspace?

No. Only the configured file bar.c will be updated to the latest version. Since the src directory is partially configured, new files under it won’t be downloaded.

Partially configured directories are designed this way to have a way to avoid new content from being unexpectedly downloaded. If you want the update to download "anything new under a given path", ensure this path is "fully checked".

Suppose you are working on a partial workspace, and for whatever reason (like doing a merge), you need to convert it into a full workspace. How can you achieve that?

Easy.