Please wait, while our marmots are preparing the hot chocolate…
- (organisation de workshops de 2 jours)
- (git + bash + python/R + SQL)
- instructeur et contributeur
- membre du comité « Lesson Infrastructure »
- release manager
- {no}
- Vous !? {slide}
- profiles
- attentes
- *a priori*
# @copy:#agenda:
## Principe de la formation {libyli postits}
- Alternances entre
- explications théoriques
- explications en pratique (démo) {demo}
- pratique (par vous) {action}
- Support (beaucoup en anglais) {dense}
- contenu Software Carpentry
- + ajouts + slides
- Interactivité et adaptabilité {dense}
- interrompre pour demander des clarifications
- planning approximatif ⇒ poser des questions d'ouverture
- Post-its * * ** ** {libyli}
- pendant les explications
- **rose/rouge** = **problème** (contenu, rythme, ...)
- pendant la pratique
- *jaune/vert* = travail *terminé*
- **rose/rouge** = appel **à l'aide**
- avant les pauses, écrire sur les post-it et les rendre
- points à *conserver* et à **améliorer**
## Pour mieux vous connaître {no-print libyli}
- Qui est familier avec la gestion de versions ?
- Qui utilise un outil de gestion de versions ?
- Qui est à l'aise avec {libyli}
- la notion d'arborescence de fichiers ?
- la notion de ligne de commande ?
- le terminal ?
- l'interpréteur bash ?
- les scripts bash ?
## Fichiers, Ligne de commande et Bash
- Référence : cours complet *Software Carpentry*
- [http://swcarpentry.github.io/shell-novice/reference/](http://swcarpentry.github.io/shell-novice/)
- Notion de fichier texte (brut) {slide}
- le texte est sauvé tel quel dans le fichier
- parfois de la couleur est ajoutée (uniquement à la visualisation)
- exemples : fichiers `.txt`, `.R`, `.Rmd`, `.md`, `.py`, `.html`, …
- éditeurs de texte : emacs, vi, nano, **atom**, RStudio
- Text brut ≠ Word/Libreoffice/… {slide}
- text riche
- avec style et mise en page
- sauvegarde de beaucoup plus que le texte (fichier binaire)
## Pratique
- Démo *{demo}*
- `pwd`, `ls`, `cd`, `mkdir`
- éditeur de texte
- `cp`, `mv`, `rm` (attention, pas de corbeille)
- `find`, `tree`
- À vous *{action}* {slide}
- se connecter au cluster de calcul
- déterminer le chemin du dossier courant
- créer un dossier `formation-git`
- se déplacer dedans
- y créer/modifier un fichier `notes.md` avec quelques lignes
- utiliser le plus possible l'autocomplétion (touche "tab" ↹)
## Gestion de version avec « Git »
- Référence : leçon *Software Carpentry*
- « git-novice »
- [http://swcarpentry.github.io/git-novice/](http://swcarpentry.github.io/git-novice/)
- Référence : présentation « courte » par votre formateur
- « git » et « gitlab »
- [http://home.heeere.com/research.html#twitwi/Presentation-2016-09-01-git-gitlab-hcurien](http://home.heeere.com/research.html#twitwi/Presentation-2016-09-01-git-gitlab-hcurien) {dense}
# @copy:#plan:
# @copy:#plan: %+class:inred: .vcs
## Why? {*no-status image-fit bottom-left darkened}
## Version Control: What? {libyli}
- A version control system (VCS)
- records what you and your collaborators have done
- allows easy replication across machines
- allows you to easily see changes
- allows you to easily experiment with new things
- Why dropbox/google drive/... is not sufficient
- safety of your data
- ownership of your data
- semantics of your changes
- Why CVS/SVN (Subversion) might not be sufficient
- centralized nature of SVN: a single server acts as a repository
- working in the train/plane/countryside
- speed limit
- {no densequote} ## Git {libyli} -SVN → Git migration in progress. 8h to retrieve full SVN history,
less than 1min to push full history to Git (same network)!
{densequote} - History of Git - open source - initiated by Linus Torvalds - first release: 7 April 2005 - version 2.11.0 on the 29th November 2016 - fast and efficient - most used version control system # @copy:#plan: %+class:inred: .git ## Setting up / Customizing Git {#introduceyourself libyli} - Introducing yourself (MANDATORY)Git (/ɡɪt/) is a version control system (VCS) for tracking changes in computer files and coordinating work on those files among multiple people. It is primarily used for software development, but it can be used to keep track of changes in any files.
wikipedia
As a distributed revision control system it is aimed at speed, data integrity, and support for distributed, non-linear workflows. Git was created by Linus Torvalds in 2005 for development of the Linux kernel, with other kernel developers contributing to its initial development…
git config --global user.name "John Doe"- these are used to “sign” your changes - the email is not used to send email and can be fake - Choose your editor
git config --global user.email john@doe.com
git config --global core.editor "emacs"- The global configuration is in **~/.gitconfig** - Software Carpentry help about configuring git - http://swcarpentry.github.io/git-novice/02-setup/ - Fancy colors and shortcuts
git config --global color.ui true## Setting up git *{action}* *{demo}* {no-print}
git config --global alias.st status
git config --global alias.ci commit
git config --global user.name "John Doe"- To choose emacs as editor
git config --global user.email john@doe.com
git config --global core.editor "emacs"- To choose nano as editor
git config --global core.editor "nano -w"- To choose atom as editor
git config --global core.editor "atom --wait" {dense}
- Trapped in an editor?
- save and quit "vi" by typing `:wq` Enter
- save and quit "nano" with Ctrl+O Enter Ctrl+X
- save and quit "emacs" with Ctrl+X Ctrl+S Ctrl+X Ctrl+C
## Starting with Git {libyli}
- Initializing your repository/project (the current directory) git init- What's up? (this does not modify anything)
git status- Deciding what is relevant
git add file1 file2 …@SVG: swc-media/git-staging-area.svg 800 230 {slide unpad} ## Adding and changing some files *{demo}* // script "simple.txt"
git commit
cp -r base mypaper ; cd mypaper git init git status git add mypaper.tex cvpr.sty git status git commit git status ... and more… - (diagram) {denser} ## Creating a repository *{action}* - Create a folder `recipes` - Change the current directory (go inside `recipes`) - Init your repository - Run `ls -a`, what does it list? - (draw your diagram) {dense} ## Adding and changing a file *{action}* - Create a `salad.md` file and type a recipe in it - See what `git status` gives - Add your file and check `git status` again - Commit your file, and check `git status` again - Modify your recipe and commit your changes, … repeat - (draw your diagram) {dense} - Recap - beginning
git init- working
git add ...
git commit [-m ...]
git status## Finer inspection and control over changes {libyli} - Keep your project clean: ignoring files - **.gitignore** file(s) - patterns - **blabla.* ** (ignore files starting with `blabla`) - **!blabla.my_precious** (but not `blabla.my_precious`) - **/build/** (ignore the `build` folder) - **build/** (ignore all `build` folders (at any level)) - ***~** (ignore files ending with `~`) - What did I just modify?
git add ...
git commit [-m ...]
git status- What happened?
git diff [...]
git log## Adding and changing a file *{action}* - Create a `tmp.log` file with some useless content in it - Create a `all.log` file with some useless content in it - Check `git status` - Create a `.gitignore` with one line containing `tmp.log` - Check `git status` - Commit the `.gitignore` file - Check `git status` # @copy:#plan: %+class:inred: .githisto ## Exploring the History {libyli} - Remember **git log**? - Each commit is written in stone - parent(s) commit - author - modifications - sha1sum (e.g. cb6dc3cb1f4f5eb15c1d9b2b25ae741cd73c0554)
- Can be diff'ed against
git diff cb6dc3...- Can be use to retreive a file
git checkout cb6dc3... -- thefile.ext- … get commit IDs, diff, checkout *{demo}* ## Exploring history *{action}* - Find the SHA1 of a previous version (your choice) - Diff with an old version of your recipe - Recover an old version - Recover the latest version - NB: call for help in case of “detached HEAD” - run `git checkout master` # Good Files and Good Commits
*You're writing history! do it well!{denser}* {no-print} ## What to commit, what not to commit? - Commit {libyli} - the `.gitignore` file(s) - any "source" file (something typed by a human) - Avoid committing {libyli} - binary files that change often (NB: word/excel/... are binary) - generated files (that can be regenerated in a reasonable time) - temporary files - any file that is big - see the "[git lfs](https://git-lfs.github.com/)" (large file storage) extension for that - Tip {slide} - if you have personnal habits (or editor conventions) - e.g. "files ending in `~` are temporary" - use a global ignore file - put the pattern in a file (outside your repository) - register this file as your personnal ignore file, with -
git config --global core.excludesfile /pathto/yourfile{no} ## What is a good commit? {libyli} - You're writing history! do it well! - Use conventional commit messages {libyli} - first line is the short message (50 character) - the rest is separated from the short message by an empty line - be consistent on the short message format, e.g. - start with a capital letter - do not put a `.` at the end - recommended reading on commit messages - Use meaningful commit messages, e.g. {libyli} - `Changes{dense}` (bad, unspecific) - `Add line "if attr is None: return False"{dense}` (bad, redundant) - `Handling problem with unset attributes{dense}` (ok, ✓) - Using meaningful content - don't make huge commits with unrelated changes - split changes in as many "independent" commits as needed ## Tips and Tricks (better and lost commits) {libyli} - Fine-grained selection of the parts to add - use the interactive `git add -p yourfile.ext` - Fix the previous commit - make the changes (corrections) - use `git commit --amend` - can change the message too `git commit --amend -m '.....'` - ok *{demo}* # @copy:#plan: %+class:inred: .multigit ## Back to the Future: parallel universes *{demo}*
git log git checkout 41474a33e098689b... emacs paper.tex git commit git log git log --graph --oneline --all --color git checkout -b mytest # Get a nice summary git log --graph --oneline --all --decorate # GUI gitk gitk --all## Branches: recap {libyli} - The many faces of `git checkout` 😓😞😢 - moving in the history
git checkout sha1-or-branch-name{denser} - creating a new branch at current position
git checkout -b new-branch-name{denser} - getting a file from the history
git checkout sha1-or-branch-name -- filename.ext{denser} - Branch - a label for a commit - automatically follows on new commit (**git commit**) - `master` is a branch created by `git init` (by convention) - `HEAD` is a reference to the last checked out revision - Use of “sha1” or branch-name (e.g. brrrr) - Shortcuts (about ^ and ~)
cb6dc3, brrrr, HEAD,## Parallel universes and branches *{action}* - Use `git log` to find the SHA1 of a previous commit - Use `git diff` to compare your version with this one - Use `git checkout yourSha1` to get this version - Check that the files have changed - Use `git checkout -b magic` to create a new branch there - Make new commit(s) - Check the history - Use `git checkout master` to switch back to "master" - Check the history and current version
HEAD^, HEAD~, HEAD~~, HEAD~2, HEAD~42,
HEAD^2, cb6dc3^42, tagggg
`git log --graph --oneline --all --decorate` ## Merging versions - Merging 41474a33… into the current version
git merge 41474a33- Possible situations {slide libyli} - already the same version (or 41474a33… is an ancestor) - → nothing {no} - the current version is an ancestor (older) - → “fast-forward” merge {no} - versions have diverged - → full merge {no} - conflicts only if the same file has been modified at the same place - Merging branch `brrrr` into master *{demo}*
git checkout master{slide} ## Merging: recap - Always commit before merging - commit is cheap, easy and local - you never loose anything when merging // maybe your temper - Non-conflicting **git merge** ⇒ automatic commit - Conflicting **git merge** - (partial merge) - non-conflicting changes are automatically added - you: solve conflict - you: `git add` - you: `git commit`
git merge brrrr
- Exploring history - **git log** - **gitk [--all]** - **git log --graph --decorate --oneline --all --color** ## Different types of merge *{action}* - Switch to the `master` branch - Create a branch named `wip` (work in progress) - Using several commands - create a situation where `master` is ahead of `wip`
(`wip` is an ancestor of `master`) - from master merge the content of `wip` ⇒ "Already up-to-date" - And - create a situation where `wip` is ahead of `master` - from master merge the content of `wip` ⇒ "Fast-forward" - And - create a situation where `wip` and `master` diverged, such that - some part of the file(s) is touched by a single branch - some part of the file(s) is touched by a both branches - from master merge the content of `wip` ⇒ "Conflict" - solve the conflict # @copy:#plan: %+class:inred: .gitlab ## What is GitLab (and GitHub) - GitLab - a company providing support and advanced features - an open source project (Community Edition) - a web application - collaboration platform - hosting git repositories - visualizing repositories - managing issues/tickets -
{no densequote} ## Let's Go - Create a repository on GitLab - Push our content - link our repository to the remote repository (on GitLab) - push the changes to this remote repository - On another machine - clone the repository - make changes, commit and push them - On this machine - pull changes: fetch them and then merge ## Recap GitLab (and Git remotes) {libyli} - GitLab project == git repository (+ wiki etc) - The local repository can be linked to some remote one(s) - When we `git clone`, a remote named `origin` is set up - When we `git fetch`, we download the remote changes - ... we can then use `git merge` to integrate them - ... or `git pull` that does fetch+merge - When we `git push`, we upload our history of changes to the remote - Reminder: always commit before merge or pull ## More GitLab (additions to git) {libyli} - Groups - groups of users (e.g., PhD student and supervisors) - automatic access to the projects of the group - Forking - take a repository on GitLab - make a “personal” copy of this repository (still on GitLab) - Merge requests (pull requests in GitHub) - ask for another repo to integrate changes from your fork - Issues - bug - questions - feature requests - Wikis - set of pages, in markdown - (also accessible as a git repository) # @copy:#plan: %+class:inred: .workflows ## Git workflow: repositories and branches - Shared central repository (for small tight groups) - one repository on gitlab - all collaborators have read/write access - a "master" branch, with possible additional branch - Set of personal "central" repositories - one "fork" on gitlab for each collaborator - the fork acts as the collaborator's branch - merging into the "principal" repository via "merge requests" (or pull requests) - Fully distributed (rarely used today) - repositories on collaborators machines - **fetch** from one another ## Git workflow: merging versus rebase {libyli} - Merging - `git merge` - keeps explicit information about parallel work - creates ugly history with several active collaborators - Rebasing - `git rebase` - replays the diverging commits on top of latest ones - creates a more linear history - warning: rewrites history - Recommendation - start with merging - when you get annoyed by the shape of the history - if you're used to merge/push/pull - then start using `git rebase -i origin/master` - *{demo}* *{action}* ## Advantages and dangers ofGitLab offers git repository management, code reviews, issue tracking, activity feeds, wikis.
changing history {libyli} - Advantages - cleaner, more readeable history (`rebase`) - possibility to correct mistakes - commited generated files - too many files in a commit - typo in message - Dangers - rewriting history creates a new history - if the history is public (big) problem on pull/merge - lost "merge" semantics # Best Practices
*collaborating with yourself and others{denser}* {no-print} ## Best Practices {libyli} - commit early and often - always commit before merge (or pull) - reminder: use meaningful commit messages - first line is the short message (50 character) - recommended reading on commit messages - reminder: avoid committing - binary files that change often (NB: word/excel/... are binary) - generated files (that can be regenerated in a reasonable time) - temporary files - keep your git status clean - don't put git repositories inside git repositories - more ## Best Practices for
Latex/Markdown/HTML/… {libyli} - Reviewing your changes - check that it "compiles" and use git status (e.g., for missing images) - use git diff (and e.g. latexdiff) - use a spellchecker as you type - Normalizing your writing - Goals - easier search - easier reading - better versioning - the “one sentence per line” rule - never put more than one sentence on one line - avoid as much as possible to break a sentence on multiple lines (e.g., don't reflow/justify your file) - remove spaces at end of lines - remove all " " (double space, makes search difficult) # @copy:#plan: %+class:inred: .concl ## Tools for Git - GUI - Bundled with git: `git gui` and `gitk` // maybe not anymore - Many others (gitg, qgit, GitX, tortoisegit, Netbeans, ...) - graphical user interfaces for Git - huge list of frontends and tools - Help for installation - for Windows, use "gitbash" - details and video helper in the software carpentry instructions ## Key Points and More {deck-status-fake-end} - Version control - keep track of what happened - store semantic snapshots/versions of your “code” - Git - “distributed” version control: you have a complete repository - efficient and widely used - one repository per project - GitLab : a place to share repositories (projects) - visualization of the repositories, wiki pages, issue tracker, … - groups of users (e.g., PhD student and supervisors) - More links - interactive learning of branching in Git - official website - graphical user interfaces for Git - for Git by a git, ask Linus Torvald - Pro Git book (available online) - git-tips ## Correspondence git ⇄ svn {libyli} - git commit ⇄ none - git commit ; git push ⇄ svn commit - git fetch ⇄ none - git fetch ; git merge ⇄ svn update - git pull == git fetch ; git merge - {no} - NB: you can also use git to collaborate with SVN users, using "git svn" ## Going further {no-print} - git remote add - git remote set-url - git tag - git rebase - git commit --amend - git reflog - git ls-files - git cherry-pick - git revert - git bisect
/ − − will be replaced by the title
