README.md plus jolis v2

This commit is contained in:
eleonore12345 2024-07-25 11:00:44 +02:00
parent 28f6a4f58b
commit aacf9d14b0
4 changed files with 47 additions and 7 deletions

View File

@ -1,4 +1,5 @@
# Overview # Overview
git_update.sh is a bash script performing a punctual synchronization of a git repository. It can either be a new clone or an update. git_update.sh is a bash script performing a punctual synchronization of a git repository. It can either be a new clone or an update.
*CAREFUL*: git_update.sh is not working-directory safe and can delete your work. *CAREFUL*: git_update.sh is not working-directory safe and can delete your work.
``` ```
@ -7,11 +8,14 @@ git_update.sh -d ~/website -r V3 $REMOTE_URL
Each time this command is called, the ~/website directory is resync with corresponding git branch or tag V3. Each time this command is called, the ~/website directory is resync with corresponding git branch or tag V3.
# Installation # Installation
Download the git_update.sh file. You can place it in /usr/local/bin. Download the git_update.sh file. You can place it in /usr/local/bin.
Dependencies: Bash and Git Dependencies: Bash and Git
# Use # Use
## Help extract ## Help extract
``` ```
NAME NAME
git_update.sh git_update.sh
@ -34,24 +38,29 @@ DESCRIPTION
This script will replace the destination with the wanted commit of a git repository. The history is not preserved but tags are. Untracked files remain. This script will replace the destination with the wanted commit of a git repository. The history is not preserved but tags are. Untracked files remain.
The git commands have been chosen so as to minimize the memory and bandwidth usages. The git commands have been chosen so as to minimize the memory and bandwidth usages.
``` ```
## Cloning ## Cloning
git_update.sh will only clone in an empty repository, by default the current working directory. git_update.sh will only clone in an empty repository, by default the current working directory.
If called in a repository with a .git directory, it will update (see below). If called in a repository with untracked files, it will fail. If called in a repository with a .git directory, it will update (see below). If called in a repository with untracked files, it will fail.
Only the commit and the necessary objects will be cloned. The commit can be indicated through a reference, either a tag or branch, otherwise the last commit of main is the default. Only the commit and the necessary objects will be cloned. The commit can be indicated through a reference, either a tag or branch, otherwise the last commit of main is the default.
*CAREFUL*: the command git branch will always show the original branch name and will not follow the eventual branch changes. *CAREFUL*: the command git branch will always show the original branch name and will not follow the eventual branch changes.
git_update.sh clones using git_update.sh clones using
" git clone --recurse-submodules --shallow-submodules --depth=1" `git clone --recurse-submodules --shallow-submodules --depth=1`
## Updating ## Updating
git_update.sh will update if the repository already contains a .git. Untracked files and directories will be kept. Any local modification to a tracked file or created tracked file will be deleted in favor of the new commit. The history will only contain the very last commit. git_update.sh will update if the repository already contains a .git. Untracked files and directories will be kept. Any local modification to a tracked file or created tracked file will be deleted in favor of the new commit. The history will only contain the very last commit.
git_update.sh updates using git_update.sh updates using
" git fetch --tags --depth=1 --prune --prune-tags --force origin $ref ```
git fetch --tags --depth=1 --prune --prune-tags --force origin $ref
git reset --hard --recurse-submodules FETCH_HEAD git reset --hard --recurse-submodules FETCH_HEAD
git submodule update --init --recursive --force --depth=1 --remote git submodule update --init --recursive --force --depth=1 --remote
git reflog expire --expire=now --all git reflog expire --expire=now --all
git gc --prune=now [--aggressive]" git gc --prune=now [--aggressive]"
```
## Examples ## Examples
A) Cloning the last commit of a branch into a non-existing directory A) Cloning the last commit of a branch into a non-existing directory
`git_update.sh -r myBranch -d myDirectory $REMOTE_URL` `git_update.sh -r myBranch -d myDirectory $REMOTE_URL`
@ -64,6 +73,7 @@ git gc has been called with the --aggressive option, hence the objects in the .g
CAREFUL: git branch still displays the original branch name and not the one of the tag. CAREFUL: git branch still displays the original branch name and not the one of the tag.
# Development process # Development process
git_update.sh has been written by the French association Jean-Cloud (https://www.jean-cloud.net), in the process of developing Shlagernetes, an orchestration tool. git_update.sh has been written by the French association Jean-Cloud (https://www.jean-cloud.net), in the process of developing Shlagernetes, an orchestration tool.
Shlagernetes allows storing services on fallible second-hand servers and tries to consume the less possible resources. Shlagernetes allows storing services on fallible second-hand servers and tries to consume the less possible resources.
@ -72,9 +82,11 @@ You can find them under /test/performance_tests.
Please refer to /doc/development_explanations for more information about the tests and choices made in the elaboration of git_update.sh. Please refer to /doc/development_explanations for more information about the tests and choices made in the elaboration of git_update.sh.
# Testing # Testing
The script test_git_update.sh can be found at /test/functional_tests. It will create a repository to test on before testing. The script test_git_update.sh can be found at /test/functional_tests. It will create a repository to test on before testing.
Please refer to the readme of test and the readme of test/functional_tests. Please refer to the readme of test and the readme of test/functional_tests.
# How to contribute # How to contribute
If you have any suggestion or question about git_update.sh, please contact us at contact@jean-cloud.org. If you have any suggestion or question about git_update.sh, please contact us at contact@jean-cloud.org.
We would be delighted to hear your feedback. Feel free to send diff files with improvement suggestions. We would be delighted to hear your feedback. Feel free to send diff files with improvement suggestions.

View File

@ -1,12 +1,16 @@
# Introduction # Introduction
performance_tests contains tests who compare the memory and bandwidth usage of different Git commande in order to compare them. performance_tests contains tests who compare the memory and bandwidth usage of different Git commande in order to compare them.
functional_tests contains tests who verify that git_update.sh is behaving as expected. functional_tests contains tests who verify that git_update.sh is behaving as expected.
Both rely on the same test repository that is created by test_repo_creation.sh. Both rely on the same test repository that is created by test_repo_creation.sh.
Both have to be run from their directory because they rely on relative paths. Both have to be run from their directory because they rely on relative paths.
# Creation of the test repository # Creation of the test repository
The script test_repo_creation.sh creates a remote/$REPO_NAME repository. The script test_repo_creation.sh creates a remote/$REPO_NAME repository.
## Help extract ## Help extract
``` ```
NAME NAME
test_repo_creation.sh test_repo_creation.sh
@ -20,34 +24,48 @@ OPTIONS
-l [link] adds the linked online repository as remote and pushes changes to it. Must be en empty repository. -l [link] adds the linked online repository as remote and pushes changes to it. Must be en empty repository.
-s creates a submodule remote/$SUB_NAME and includes it in remote/$REPO_NAME. -s creates a submodule remote/$SUB_NAME and includes it in remote/$REPO_NAME.
``` ```
# Description of the created repository # Description of the created repository
Here is a history of the commits: Here is a history of the commits:
### branch main ### branch main
commit sample0 1M created commit sample0 1M created
commit sample1 1M created commit sample1 1M created
commit sample3 5M created commit sample3 5M created
commit sample4 1M created commit sample4 1M created
commit sample3 5M deleted (tag : tagging_point) commit sample3 5M deleted (tag : tagging_point)
[if -s is selected] commit adding $SUB_NAME module [if -s is selected] commit adding $SUB_NAME module
### branch secondary ### branch secondary
commit sample0 1M created commit sample0 1M created
commit sample1 1M created commit sample1 1M created
commit sample2 500K created commit sample2 500K created
Which gives the latest state: Which gives the latest state:
### branch main ### branch main
sample0 1M sample0 1M
sample1 1M sample1 1M
sample4 1M sample4 1M
### branch secondary ### branch secondary
sample0 1M sample0 1M
sample1 1M sample1 1M
sample2 500K sample2 500K
If the -s option is selected, the submodule is created, and integrated in the last commit of main, with this history of commits: If the -s option is selected, the submodule is created, and integrated in the last commit of main, with this history of commits:
### branch main ### branch main
commit first 1M sample created commit first 1M sample created
which gives the latest state: which gives the latest state:
### branch main ### branch main
sub_sample0 sub_sample0

View File

@ -1,13 +1,16 @@
## Introduction ## Introduction
test_git_update.sh performs functional tests on git_update.sh to verify that it is behaving as planned. test_git_update.sh performs functional tests on git_update.sh to verify that it is behaving as planned.
## Tests ## Tests
The script consists in nine tests. The three first ones test the cloning and the following test the updating of git_update.sh. The script consists in nine tests. The three first ones test the cloning and the following test the updating of git_update.sh.
Each test consists of calling git_update.sh, eventually after some modifications, and compare the result with the expected result. The comparison is done by *_check functions, which modify each a boolean *_result. Each test consists of calling git_update.sh, eventually after some modifications, and compare the result with the expected result. The comparison is done by *_check functions, which modify each a boolean *_result.
If all booleans of checks conducted are true, then the boolean case* is also true and the test is a success. If all booleans of checks conducted are true, then the boolean case* is also true and the test is a success.
## Help extract ## Help extract
``` ```
NAME NAME
test_git_update.sh test_git_update.sh
@ -30,6 +33,8 @@ DESCRIPTION
TEST8: git updated, before and after changing a tag, deleting and adding files in the process TEST8: git updated, before and after changing a tag, deleting and adding files in the process
TEST9: git updated fast-forward on submodule on main" TEST9: git updated fast-forward on submodule on main"
``` ```
## Debugging ## Debugging
In case of problem, it is adviced to run test_git_update.sh -c to clean the repository. In case of problem, it is adviced to run test_git_update.sh -c to clean the repository.
This will force test_repo_creation.sh to run again and provide a new test repository. This will force test_repo_creation.sh to run again and provide a new test repository.

View File

@ -1,9 +1,11 @@
## Introduction ## Introduction
Our objective is to find the least-consuming method in terms of memory and bandwidth resources. Our objective is to find the least-consuming method in terms of memory and bandwidth resources.
We are interested in cloning one specific state of the repository. We are not interested in its history or the possibility to change it from the server where it has been cloned. We are interested in cloning one specific state of the repository. We are not interested in its history or the possibility to change it from the server where it has been cloned.
The tests rely on a repository created by test_repo_creation.sh. It measures memory and bandwidth usage for different git commande. The tests rely on a repository created by test_repo_creation.sh. It measures memory and bandwidth usage for different git commande.
## Tests ## Tests
The script consists of twenty-nine tests (listed in the results below), based on three functions: generate_random_file, get_storage_used and get_bandwidth. The script consists of twenty-nine tests (listed in the results below), based on three functions: generate_random_file, get_storage_used and get_bandwidth.
generate_random_file uses the bash command dd and /dev/random. generate_random_file uses the bash command dd and /dev/random.
@ -14,6 +16,8 @@ The first five tests concern cloning.
The following tests involve updating the repository using different commands, with three cases for each command: after adding a file, after deleting a file, after adding then deleting a file. The following tests involve updating the repository using different commands, with three cases for each command: after adding a file, after deleting a file, after adding then deleting a file.
## Help extract ## Help extract
```
NAME NAME
performance_tests.sh performance_tests.sh
SYNOPSIS SYNOPSIS
@ -63,6 +67,7 @@ DESCRIPTION
TEST27: --depth=1 fetching+merging -s ours with reflog and gc after addition of a 1M file TEST27: --depth=1 fetching+merging -s ours with reflog and gc after addition of a 1M file
TEST28: --depth=1 fetching+merging -s ours with reflog and gc after removal of a 1M file TEST28: --depth=1 fetching+merging -s ours with reflog and gc after removal of a 1M file
TEST29: --depth=1 fetching+merging -s ours with reflog and gc after addition then removal of a 1M file" TEST29: --depth=1 fetching+merging -s ours with reflog and gc after addition then removal of a 1M file"
```
## To go further ## To go further
To learn more about the process of testing different git methods conducted, please refer to doc/developement_explanations.md. To learn more about the process of testing different git methods conducted, please refer to doc/developement_explanations.md.