The four main commands are
cln init-- initialize a new project
cln prepare-- prepare an action
cln run-- run an action
cln remove-- remove an action
Check the help screen for usage info:
$ cln help Computational Lab Notebooks cln SUBCOMMAND CLI to help you manage a lab notbook with git and git-annex. === subcommands === init Initialize a new project prepare Prepare an action remove Remove a pending action run Run an action version print version information help explain a given subcommand (perhaps recursively)
Initialize a new project
To initialize a new project/lab notebook use
cln init. The
app is a wrapper on
git-annex, so your project lives in a
certain directory. That directory is the one in which you run the
cln init command!
There is nothing magical about the
cln init command. It sets up the
required directories, makes a
README.md file, and then sets up a new
After you run
cln init, you are ready to start getting some real
To view the usage screen, run
cln help init or
cln init -help.
$ cln help init Initialize a new project cln init PROJECT_NAME === info === Initialize a new computational lab notebook project using git and git-annex. For more info, see https://mooreryan.github.io/cln_docs/ === flags === [-help] print this help text and exit (alias: -?)
First make a new directory and
cd into it.
$ mkdir my_project && cd my_project
Then initialize a new project called
$ cln init 'My Project'
Notice how I put the
My Project bit inside single quote characters
')? If your project name has spaces, you need to remember to quote
- If you run
cln initin a directory that already has a git repository, you will get an error.
Prepare an action
To prepare an action or command to run, use the
cln prepare command.
This creates a new action file (i.e., bash script) and a git commit
template in the
When I say "action", I just mean, something (e.g., a shell script) that will change the state of the repository (aka your lab notebook). The reason we formalize these things as actions is that it makes it a lot easier to figure out what happened in your project/repository a couple of months down the line when you come back to it.
To vier the usage screen, run
cln help prepare or
$ cln help prepare Prepare an action cln prepare ACTION === info === Generates an action file and a git commit template in the '<project_root>/.actions/pending' directory. For more info, see https://mooreryan.github.io/cln_docs/ === flags === [-help] print this help text and exit (alias: -?)
Prepare a shell command to run:
$ cln prepare 'echo "hello, world" > hello.txt'
Rather than running
echo "hello, world" > hello.txt yourself,
"preparing" the command sets it up so that you have a way to run the
command that makes it easier to remember what you did a few
months/years down the line.
What files were created?
You have the action file (aka bash script):
$ cat .actions/pending/action__262765981__2021-04-09_14:54:48.sh /usr/bin/echo "hello, world" > hello.txt
The first weird number in the file name is a hash code representing the contents of the file. The second is the date and time in which you prepared the command.
And you have the git commit template. Git commit templates are a nice way to help you write good commit messages with as little annoyance as possible.
$ cat .actions/pending/action__262765981__2021-04-09_14:54:48.gc_template.txt PUT COMMIT MSG HERE. == Details == PUT DETAILS HERE. == Command(s) == /usr/bin/echo "hello, world" > hello.txt == Action file == action__262765981__2021-04-09_14:54:48.sh
Notice how the prepared action and the action file are automatically included in the commit template? This is really helpful for when you're going back searching through the commit logs. It will give you context of the exact command that was run connected with the exact changes it produced in any of the files included in your repository.
After you run the action, add the changes to git and then go to actually commit the changes, you can use the commit template like this:
$ git commit -t .actions/pending/action__262765981__2021-04-09_14:54:48.gc_template.txt
This will open up a text editor with the contents of that file. Then you just have to edit the summary message and the details. For the summary, just put a short < 50 character explanation of what you did. For the details, try and put as much info and context as you will need to figure out what you did and why you did it. Basically, you should put the sort of things here that you would be putting in your lab notebook anyway.
If you try and prepare an action when there is already a pending action, you will get an error. It's set up this way to encourage you to set up a single action, run it (which then changes the state of your repository, then commit those changes to the repository. So it's like 1 action <=> 1 commit.
Run an action
To run an action, use the
cln run command.
This will look for a valid pending action, and if it finds one, runs
bash. If it succeeds, then the program will also move the
action into the
- You will get an error if there are no pending actions.
- If there are more than one pending action, you will also get an error.
cln preparecommand won't let you make more than one pending action.
- But even if you make an extra "by-hand" with the format exactly
cln runcommand still won't let you run it if there are more than one.
To do a "dry run", i.e., to have the
cln prepare program just tell
you what it will do without actually running any actions, you can use
-dry-run flag. Here is an example:
$ cln run -dry-run
You will see some useful output. Check it out and make sure it looks good. If so, then you can run the action.
-dry-run is totally optional, I recommend that you do a
dry run before running the action so you can check that you haven't
made any obvious mistakes!
A note on exit codes
To determine wether an action has failed or succeeded, we check the exit codes.
- Exit code of
- Any other exit code means failure.
Let's say that you prepared an action to run some bash script that you have written like this:
printf "Starting script!\n" ## This command will fail. cat this_file_doesnt_exist.txt > new_file.txt ## But the script will keep going and run this, which will succeed. printf "All done!\n"
If you ran this at the command line with bash, you would get this:
$ bash silly.sh Starting script! cat: this_file_doesnt_exist.txt: No such file or directory All done!
Do you think it "succeeded"? Let's check the exit code.
$ echo $? 0
According to the exit code, yes, the script as a whole succeeded. But
if you check the contents of
new_file.txt you will see that it is
empty, which is probably not what you wanted! So what happened is
that an intermediate command failed, but given the way the script was
written, the script as a whole succeeded.
How to fix it
This is something you will want to watch out for. If this were run as
an action by
cln run, the
cln program would consider this program
a success and move the action into the
If you have scripts like this that run lots of commands, you should consider breaking them up. That way you stick to the one action, one commit principal mentioned above.
If you really want a multi-command bash script
In some cases, you may need more than one command, e.g., when
something only makes sense if something succeeds first. In that case,
you can join the commands with
&&. Let's try that on the previous
printf "Starting script!\n" && \ cat this_file_doesnt_exist.txt > new_file.txt && \ printf "All done!\n"
And run that.
$ bash silly2.sh Starting script! cat: this_file_doesnt_exist.txt: No such file or directory
See how the last
printf command was not run? Let's check the exit
$ echo $? 1
That's a failing exit code. So, the
cln run program would consider
that script to have failed.
Remove an action
Sometimes you need to remove pending actions:
- You made a mistake when prepping the action.
- You don't need that action anymore.
- There was an error when running the action.
The point is that you sometimes will want to delete actions! The way
cln program currently works is that you can only ever have
one pending action at a time. So if you have messed something up and
need to get rid of the action, you will need to remove it. To do
this, we use the
cln remove program.
Deleting, failing, or ignoring actions
There are three ways to "remove" a pending action:
- Deleting it
- Marking it as "failed" (aka failing the action)
- Marking it as "ignored" (aka ignoring the action)
Of these three only deleting truly deletes the action data. (And even then, if you've already checked it into your git repository you will still be able to get it back.)
"Failing" an action means moving the action (and its associated git
commit template) out of the
.actions/pending directory and into the
"Ignoring" an action is similar to "failing" an action except that the
pending action is moved to the
.actions/ignored directory rather
The reason for this distinction is to help you when you're going back through your history and commit logs. I.e., it will give you more context as to why a pending action was no longer necessary.
Specifying the method
You specify the method you want to use with the
-method flag like
cln remove -method delete
cln remove -method fail
cln remove -method ignore
Saving the results in the git repo
After you run the
cln remove command you will see some output
suggesting how to proceed. You may see something like this:
~~~ * Check which files have changed: ~~~ $ git status ~~~ * Add actions and commit templates (I know it says add--it means "track this change with git"): ~~~ $ git add .actions ~~~ * After "adding" files, commit changes: ~~~ $ git commit
As you can see, you still need to let
git know that you have moved
the actions around and commit the changes.
Manually removing actions
Like many of the
cln commands, you could just do this "by-hand".
cln remove command is a pretty thin wrapper around
what you would probably do by hand, but it does give you some nice
messages on what you may want to do next and it does some sanity
checking on your pending actions to make sure nothing has gotten crazy
in the meantime.