CmdRunner: The Plastic SCM API

When we build Plastic SCM, we add features asked by users, customers and our internal timeline, so there are some features that we have no time and resources to do. Luckily thanks to the community, we have been able to develop more plugins and features to it.

One of the ways of extending Plastic SCM is by using CmdRunner, which is is an automation layer built on top of the command line. It's built on .NET using C# and is publicly available on our Github repository, so you can fork and contribute.

Plastic SCM's command line has parameters such as --machinereadable and --format, that allow us to customize the output of the tool, so that it can easily be parsed by external tools. The plugins for Eclipse, IntelliJ, TeamCity, Hudson / Jenkins, Crucible, Bamboo and others, are entirely based on the command line client, so there are a lot of expansion possibilities.

The best way of showing what does our wrapper layer is, in fact, showing what it can do. Here is a small example:

string cmVersion = CmdRunner.ExecuteCommandWithStringResult("cm version", Environment.CurrentDirectory);
Console.WriteLine(string.Format("The cm version is: {0}", cmVersion));

If you are used to .NET's way of launching process, you can see we have abstracted all that logic into a simple, static method, and we recover the output that you would normally get from the standard output. We can do something a little bit more serious, like listing all the repositories available in a specific server.

string server = "remoteserver:8087";
string repoList = CmdRunner.ExecuteCommandWithStringResult(string.Format("cm repository list {0}", server), Environment.CurrentDirectory);

Console.WriteLine(string.Format("{0}", repoList));

The output of this small piece of text will be something like:

Have you seen how we also handle the different arguments? As a quick note before finishing this section, you can use CmdRunner to launch any other program such as difftool or semanticmerge. If is available in the PATH you can call it directly (i.e. explorer); otherwise you can call it using the full path of the executable.

We can get started by downloading the repository from github. There are two ways of getting the code:

1. Downloading the last changes in a zip file.
2. Using git-sync (get more info here), synchronizing a local repository with this one:
   
   $ cm repository create plastic-cmdrunner $ cm sync plastic-cmdrunner@localhost:8087 git https://github.com/PlasticSCM/plastic-cmdrunner.git

If you want to extend it with your own commands, you can fork the repository, that will give you your own repository to sync, and, later, if you want to contribute back, you can do a pull request and your changes will be reviewed and merged back into the parent repository.

Once you download the package you can find a Visual Studio 2010 solution with two projects:

1. CmdRunner, which contains the library.
2. Samples, which contains all the samples that are explained in this document.

All the samples have their own main method, so you just need to go to the project properties on Visual Studio and select the sample to execute.

In this sample we are going to list all the branches available on a repository. For more info about cm find, please check out this guide.

string cmdResult = CmdRunner.ExecuteCommandWithStringResult(
    string.Format("cm find branch on repositories '{0}' --format={{id}}#{{name}} --nototal", repository),
    Environment.CurrentDirectory);

ArrayList results = SampleHelper.GetListResults(cmdResult, true);
List<Branch> branches = GetBranchListFromCmdResult(results);

foreach (Branch branch in branches)
    Console.WriteLine(branch);

This code performs a basic query on our repository. The raw output of the command should look like this:

After that, and using our sample helper, we parse and convert that output into a list of strings with the method GetListResults. Finally, for each element we generate a new Branch object.

public class Branch
{
    public string Id { get; set; }
    public string Name { get; set; }

    public Branch(string output)
    {
        string[] parsed = output.Split('#');
        Id = parsed[0];
        Name = parsed[1];
    }

    public override string ToString()
    {
        return Id + " - " + Name;
    }

    public override bool Equals(object obj)
    {
        if (!(obj is Branch))
            return base.Equals(obj);

        return ((Branch)obj).Name.Equals(Name);
    }
}

This object has now the information of the name and the server, and an equal override that can help us to compare branches in the next samples. We can extend this information with all the parameters we can get from cm find.

Based on our previous sample, and with our Branch object, we are now going to replicate the changes created from one repository to another. For replicating a branch we need the full spec of it and the destination repository. This can be done with the following code:

private static ReplicationResult Replicate(Branch branch, string repository, string secondRepository)
{
    Console.WriteLine("Replicating branch {0}", branch.Name);

    string cmdResult = CmdRunner.ExecuteCommandWithStringResult(
      string.Format("cm replicate \"br:{0}@{1}\" \"rep:{2}\"",
        branch.Name, repository, secondRepository),
      Environment.CurrentDirectory);

    return new ReplicationResult(SampleHelper.GetListResults(cmdResult, true), branch);
}

The replicate command is called, and then the result is parsed and converted to a simple class called Replication Result. The raw output of the Replicate command looks like this:

If a new branch is created on the target repository, the name will be displayed at the end. After the command is completed, the result is parsed like in the previous sample, creating a new ReplicationResult object with the result of each operation.

class ReplicationResult
{
    public long Items { get; set; }
    public Branch Branch { get; set; }

    public ReplicationResult(ArrayList cmdResult, Branch branch)
    {
        Branch = branch;
        string buffer = (string)cmdResult[1];
        Items = long.Parse(buffer.Substring(buffer.LastIndexOf(' ')));
    }
}

The replication result contains then the number of replicated items, and the associated branch. This could be extended with the rest of the items displayed before.

Finally, a small report of the overall replication is generated:

private static void PrintReplicationResult(List<ReplicationResult> resultList)
{
    Console.WriteLine("Branches replicated: {0}" , resultList.Count);
    foreach (ReplicationResult item in resultList)
        Console.WriteLine("- {0} ({1} item{2})",
            item.Branch.Name, item.Items, item.Items == 1 ? "" : "s");
}

The replication output looks like this:

Following the previous example, we will now track changes on a branch, so we can replicate it from one server to another. For this scenario, the code will simulate changes in one repository, so we get the notifications in the other one.

The first thing we need to know if there are new branches, so we find the branches on both servers, and add the ones that are only on source.

List<Branch> GetBranchesToSync()
{
    List<Branch> srcBranches = GetBranchesFromRepo(mSampleRep);
    List<Branch> dstBranches = GetBranchesFromRepo(mSecondRep);
    List<Branch> newBranches = new List<Branch>();

    foreach (Branch item in srcBranches)
    {
        if (!dstBranches.Contains(item))
        {
            newBranches.Add(item);
            continue;
        }

        if (HasChangesInBranch(item))
            newBranches.Add(item);
    }

    return newBranches;
}

The next step is to find out if any of the common branches has changes. For this process the easiest option is use cm find once again.

private bool HasChangesInBranch(Branch branch)
{
    string srcResult = CmdRunner.ExecuteCommandWithStringResult(
        string.Format("cm find changeset where branch = 'br:{0}' on repositories '{1}' --format={{id}} --nototal",
          branch.Name, mSampleRep),
        Environment.CurrentDirectory);

    ArrayList srcResults = SampleHelper.GetListResults(srcResult, true);

    string dstResult = CmdRunner.ExecuteCommandWithStringResult(
        string.Format("cm find changeset where branch = 'br:{0}' on repositories '{1}' --format={{id}} --nototal",
          branch.Name, mSecondRep),
        Environment.CurrentDirectory);

    ArrayList dstResults = SampleHelper.GetListResults(dstResult, true);

    return srcResults.Count != dstResults.Count;
}

Once we have all the changesets, we compare them, and if they don't match then the branch must be replicated. All this process is displayed to the user in a balloon:

Clicking on the balloon will launch the replica process. For simulating work on other repository, there is a background thread adding items continuously and checking in, so you can just watch as the changes start showing. For this sample, thread number has been added: in the image thread 9 is the main thread and thread 10 is the background one.

In this sample we show how to convert Plastic SCM into a Dropbox-alike tray app, by tracking local file changes and commiting them automatically. One of the causes of Dropbox's success is the simplicity and the few user interaction, because most of the work is done automatically and in background.

The first thing we need to do is to configure the popout window, and getting the server information for our configuration variable. If we don't change the Configuration code, the IsSimulation, variable will be set to true, and the files will be added and changed automatically in a temporary workspace so we can just sit and watch it happen.

public CMbox()
{
    ConfigureMenu();
    string server = Configuration.ServerName;
    mSampleRep = SampleHelper.GenerateEmptyRepository(server);

    if (Configuration.IsSimulation)
        CMboxHelper.StartSimulation();
}

Once we have the configuration, we call, every 10 seconds to a function called CommitUpdates. The first thing that this function will do is to find the existing changes of the workspace, using the command cm status.

private List<Change> GetChanges()
{
    List<Change> changes = new List<Change>();
    string cmdResult = CmdRunner.ExecuteCommandWithStringResult(
            string.Format("cm status --all --machinereadable"),
            SampleHelper.GetWorkspace());

    ArrayList results = SampleHelper.GetListResults(cmdResult, true);
    for (int i = 1; i < results.Count; i++)
        changes.Add(new Change((string)results[i]));

    return changes;
}

If we use cm status with the machinereadable modifier then the output may look like this:

The generated output is converted to an arraylist as seen before, and each line is parsed into single Change element.

class Change
{
    public string Name { get; set; }
    public string FullPath { get; set; }
    public ChangeType ChangeType { get; set; }

    public Change(string resultRow)
    {
        string[] separated = resultRow.Split(' ');
        ChangeType = GetFromString(separated[0]);
        FullPath = separated[1];
        Name = Path.GetFileName(separated[1]);
    }

    private ChangeType GetFromString(string source)
    {
        switch (source.ToUpperInvariant())
        {
            case "AD+LD":
            case "LD+CO":
                return CmdRunnerExamples.ChangeType.Deleted;
            case "PR":
            case "AD":
                return CmdRunnerExamples.ChangeType.Added;
            case "CO":
                return CmdRunnerExamples.ChangeType.Changed;
            default:
                return CmdRunnerExamples.ChangeType.None;
        }
    }
}

The change contains the name of the item (for displaying in the balloon), the full path of it, and finally the change type, recovered from parsing the result.

Once the changes are recovered, and are changes that have not been checked in, a checkin operation is done, and the user is notified at the end:

private void CheckinUpdates(object sender, EventArgs e)
{
    List<Change> mChanges = GetChanges();
    if (mChanges.Count == 0)
        return;

    StringBuilder builder = new StringBuilder();
    foreach (var item in mChanges)
        builder.Append(string.Format("{0} {1}\n", item.ChangeType, item.Name));

    foreach (var item in mChanges)
    {
        if (item.ChangeType == ChangeType.Added)
            CmdRunner.ExecuteCommandWithStringResult(
                string.Format("cm add {0}", item.Name),
                SampleHelper.GetWorkspace());

        if (item.ChangeType == ChangeType.Deleted)
            CmdRunner.ExecuteCommandWithStringResult(
                string.Format("cm rm {0}", item.Name),
                SampleHelper.GetWorkspace());
    }

    CmdRunner.ExecuteCommandWithStringResult("cm ci ",
        SampleHelper.GetWorkspace());

    mTrayIcon.ShowBalloonTip(3, string.Format("{0} new changes saved", mChanges.Count),
            string.Format("The following changes have been checked in.\n{0}",
            builder.ToString()),
            ToolTipIcon.Info);
 }

In this sample we are also adding and removing items marked as added and deleted with the commands cm add and cm rm.

The notification contains the number of changes that have been checked in and a small description of each one.

In this sample we are going to use our source control as a game saver, so each time we change the level a new changeset will be created with the level info, this way we will always be able to recover our previous game status.

The game saves the status by creating a new changeset of a random file. The changeset comment contains the game status.

private void SaveGame()
{
    string item = SampleHelper.AddRandomItem();
    SampleHelper. CheckinItem(item, string.Format("{0}#{1}",
            mColors, mScore));
}

This way, for loading the games, we only need to find all changesets, filter by comment, and parse the comment. Each comment will be parsed in a new class called SaveGame, which contains the score, and the color number.

private void ReloadSavedGames()
{
    string cmdResult = CmdRunner.ExecuteCommandWithStringResult(
        string.Format("cm find changeset on repositories '{0}' --format={{comment}} --nototal",
          textBox1.Text),
          Environment.CurrentDirectory);

    ArrayList results = SampleHelper.GetListResults(cmdResult, true);

    listBox1.Items.Clear();
    results.RemoveAt(0);
    foreach (string item in results)
        listBox1.Items.Add(new SavedGame(item));

}

That content is later loaded in the game, so you can keep playing in the same status you left it.

The Load and Create New buttons allow us to create a new Plastic SCM repository for this game, or connecting to an existing repository to use saved games.

Every major feature of Plastic SCM is available on the command line. That has allowed us to create our plugins, and could be used to create a completely custom user interface. In this sample we are creating a small tool that has a pending changes view and a list of changesets. It allows us do checkin operations (including a comment) and to get which files have been changed on each commit. This is how it looks like:

This tab item allows to checkin files and add a comment.

This tab item displays the changesets, and the different changes that were done on each.

As we are simulating the behavior of normal usage, the console will output the changes that are being done automatically.

We won't focus on GUI related aspects, just in the behavior of three specific buttons, the checkin, the refresh, and the refresh from the changesets view.

This button reloads the changeset list and creates also a list of the contents of that changeset. For that we use the following commands from cm:

• find changeset (for getting the changesets with a specific format).
• log (given a specific changeset, we can get the items that have been added, moved, changed or deleted).

private void RefreshChangesetList(object sender, EventArgs e)
{
    string cmdResult = CmdRunner.ExecuteCommandWithStringResult(
        "cm find changeset --nototal --format=\"{changesetid}#{date}#{comment}\"",
        SampleHelper.GetWorkspace());
 
    ArrayList results = SampleHelper.GetListResults(cmdResult, true);
 
    changesetList.Items.Clear();
    foreach (string item in results)
    {
        Changeset cset = new Changeset(item);
        cmdResult = CmdRunner.ExecuteCommandWithStringResult(
            string.Format("cm log {0} --csFormat=\"{{items}}\" --itemFormat=\"{{path}}#{{fullstatus}}#{{newline}}\"",cset.Id),
            SampleHelper.GetWorkspace());
 
        results = SampleHelper.GetListResults(cmdResult, true);
        foreach (string changedItem in results)
            cset.Changes.Add(new Item(changedItem));
 
        changesetList.Items.Add(cset);
    }
}

The process is very similar to others, call the command, parse the result and generate new objects that will allow us to display data in the GUI.

The Changeset class is a simplified representation that contains only the id, the date, the comment and the list of changes. It also contains the code needed for parsing the output of a single cm find line.

public class Changeset
{
    public string Date { get; set; }
    public string Id { get; set; }
    public string Comment { get; set; }
    public List<Item> Changes { get; set; }
 
    public Changeset(string output)
    {
        string[] parsed = output.Split('#');
        Id = parsed[0];
        Date = parsed[1];
        Comment = parsed[2];
        Changes = new List<Item>();
    }
 
    public override string ToString()
    {
        return string.Format("{0}: cs:{1}", Date, Id);
    }
}

The Item class contains the status and the relative path to the workspace.

public class Item
{
    public string Path { get; set; }
    public string Status { get; set; }
 
    public Item(string output)
    {
        if (string.IsNullOrEmpty(output))
        {
            Path = string.Empty;
            Status = string.Empty;
            return;
        }
 
        string[] parsed = output.Split('#');
        Path = parsed[0].Replace(SampleHelper.GetWorkspace().ToLowerInvariant(),
            "");
        Status = parsed[1];
    }
 
    public override string ToString()
    {
        if (string.IsNullOrEmpty(Status))
            return string.Empty;
 
        return string.Format("{0} ({1})", Path, Status);
    }
}

The Refresh button clears both lists, gets the list of changes and filters added and deleted from changed. It uses one single command from cm: cm status. The operation done is the same GetChanges that we have done in the CMBox sample.

private void Update(object sender, EventArgs e)
{
    itemsToCommit.Items.Clear();
    itemsNotAdded.Items.Clear();

    List<Change> mChanges = GetChanges();
    if (mChanges.Count == 0)
        return;

    foreach (var item in mChanges)
    {
        if ((item.ChangeType == ChangeType.Added)
            || (item.ChangeType == ChangeType.Deleted))
        {
            itemsNotAdded.Items.Add(item);
            continue;
        }

        itemsToCommit.Items.Add(item);
    }
}

For adding "added" and "deleted" items into the list, just double-click on it.

The Checkin button will add the items that have been added to the list (added and deleted items), will do a checkin operation of the current workspace status. Finally, it will clear the comment textbox and update the list with the last changes.

private void Checkin(object sender, EventArgs e)
{
    foreach (Change item in itemsToCommit.Items)
    {
        if (item.ChangeType == ChangeType.Added)
            CmdRunner.ExecuteCommandWithStringResult(
                string.Format("cm add {0}", item.Name),
                SampleHelper.GetWorkspace());
 
        if (item.ChangeType == ChangeType.Deleted)
            CmdRunner.ExecuteCommandWithStringResult(
                string.Format("cm rm {0}", item.Name),
                SampleHelper.GetWorkspace());
    }
 
    CmdRunner.ExecuteCommandWithStringResult(
        string.Format("cm ci -c=\"{0}\"", textBox1.Text),
        SampleHelper.GetWorkspace());
 
    textBox1.Text = string.Empty;
    Update(sender, e);
}

With these few commands we can have our own Plastic SCM client, we would still need to handle errors, merges and a lot of stuff we do internally in the GUI.

All the samples shown do individual calls to the cm.exe process, but there is a faster way to archieve results, if we need to make several commands.

string repos = CmdRunner.ExecuteCommandWithStringResult("cm repository list",
    Environment.CurrentDirectory, true);

string output;
string error;
CmdRunner.ExecuteCommandWithResult(
    "cm workspace delete .",
    Environment.CurrentDirectory, out output, out error, true);

See that last boolean value? That means we are using a shell to load the commands and that will only be a single cm.exe running. If there is no shell running then a new one will be automatically launched. This would be the equivalent of the following sequence:

All the code is prepared to run just by specifying a Plastic SCM local server in the Configuration.cs. For the sake of simplicity, all the samples also create a new repository for the operation, so you can try out the code in a secure environment.

The SampleHelper includes code for:

• Creating an empty repository.
• Creating a repository with branches.
• Add and checkin of items inside the repository.
• Add and modify of items inside workspace.
• Parsing of the output, so it becomes an arraylist of strings.
• Retrieving the current temporary workspace.

This code is open source, published under an MIT license, so you can get it, modify it and customize it to your needs. We also appreciate your contributions, so you are welcome to cloning and pull requesting your own contributions.

March 22, 2019

• We replaced the references to deprecated repository administration commands like cm mkrep for their new counterparts.