links partout
This commit is contained in:
eleonore12345
parent
7708b9c19f
commit
3e1c08d4ba
10
README.md
10
README.md
@ -73,17 +73,17 @@ CAREFUL: git branch still displays the original branch name and not the one of t
|
||||
|
||||
# Development process
|
||||
|
||||
git_update.sh has been written by the French association Jean-Cloud (https://www.jean-cloud.net), in the process of developing Shlagernetes, an orchestration tool.
|
||||
git_update.sh has been written by the French association [Jean-Cloud](https://www.jean-cloud.net), in the process of developing Shlagernetes, an orchestration tool.
|
||||
Shlagernetes allows storing services on fallible second-hand servers and tries to consume the less possible resources.
|
||||
|
||||
Several tests have been performed in order to find the most energy-saving Git commands.
|
||||
You can find them under /test/performance_tests.
|
||||
Please refer to /doc/development_explanations for more information about the tests and choices made in the elaboration of git_update.sh.
|
||||
You can find them under [/test/performance_tests](./test/performance_tests).
|
||||
Please refer to [/doc/development_explanations](./doc/development_explanations) for more information about the tests and choices made in the elaboration of git_update.sh.
|
||||
|
||||
# Testing
|
||||
|
||||
The script test_git_update.sh can be found at [link to functional tests](./test/functional_tests). It will create a repository to test on before testing.
|
||||
Please refer to the readme of test [link to test/README.md](./test/README.md) and the readme of test/functional_tests [link to test/functional_tests/README.md](./test/functional_tests/README.md).
|
||||
The script test_git_update.sh can be found at [test/functional tests](./test/functional_tests). It will create a repository to test on before testing.
|
||||
Please refer to the readme of test [/test/README.md](./test/README.md) and the readme of test/functional_tests [/test/functional_tests/README.md](./test/functional_tests/README.md).
|
||||
|
||||
# How to contribute
|
||||
|
||||
|
||||
72
readme
72
readme
@ -1,72 +0,0 @@
|
||||
# Overview
|
||||
git_update.sh is a bash script performing a punctual synchronization of a git repository. It can either be a new clone or an update.
|
||||
CAREFUL: git_update.sh is not working-directory safe and can delete your work.
|
||||
|
||||
# Download
|
||||
Download the git_update.sh file.
|
||||
Bash and Git have to be installed on your computer.
|
||||
|
||||
# Use
|
||||
## Help extract
|
||||
NAME
|
||||
git_update.sh
|
||||
SYNOPSIS
|
||||
git_update.sh [-h] [-r ref] [-d dest] [-H] [-a] repository
|
||||
OPTIONS
|
||||
-h prints the help.
|
||||
-r specifies the reference to the commit to be synchronized. It can be a tag or a branch. By default, it is the last commit of branch main.
|
||||
It can be different from the actual branch. CAREFUL, the command git branch will always show the original branch name even though a branch switch has happened.
|
||||
-d specifies the destination of the clone or update. Directory must be empty if a new clone is to be made.
|
||||
If the repository to be cloned is local, and its path is passed as a relative path, the path should start from the destination.
|
||||
To avoid mistakes, absolute paths are advised.
|
||||
-H allows the $HOME directory to be used by git_update.sh. By default, git_update.sh cannot access $HOME to prevent default behavior.
|
||||
If you need the global .gitconfig located in your $HOME to be used, you should supply the -H option.
|
||||
-a specifies that the aggressive option of the git garbage collection must be used. Only advised when changes happen in many different objects. Will slow down the execution.
|
||||
DESCRIPTION
|
||||
This script will replace the destination with the wanted commit of a git repository. The history is not preserved but tags are. Untracked files remain.
|
||||
The git commands have been chosen so as to minimize the memory and bandwidth usages.
|
||||
|
||||
## Cloning
|
||||
git_update.sh will only clone in an empty repository, by default the current working directory.
|
||||
If called in a repository with a .git directory, it will update (see below). If called in a repository with untracked files, it will fail.
|
||||
Only the commit and the necessary objects will be cloned. The commit can be indicated through a reference, either a tag or branch, otherwise the last commit of main is the default.
|
||||
CAREFUL: the command git branch will always show the original branch name and will not follow the eventual branch changes.
|
||||
git_update.sh clones using
|
||||
" git clone --recurse-submodules --shallow-submodules --depth=1"
|
||||
|
||||
## Updating
|
||||
git_update.sh will update if the repository already contains a .git. Untracked files and directories will be kept. Any local modification to a tracked file or created tracked file will be deleted in favor of the new commit. The history will only contain the very last commit.
|
||||
git_update.sh updates using
|
||||
" git fetch --tags --depth=1 --prune --prune-tags --force origin $ref
|
||||
git reset --hard --recurse-submodules FETCH_HEAD
|
||||
git submodule update --init --recursive --force --depth=1 --remote
|
||||
git reflog expire --expire=now --all
|
||||
git gc --prune=now [--aggressive]"
|
||||
|
||||
## Examples
|
||||
A) Cloning the last commit of a branch into a non-existing directory
|
||||
./path/to/git_update.sh -r myBranch -d myDirectory https://git.mydomain/myname/myrepository.git
|
||||
Result: a directory myDirectory has been created in the current working directory. It is filled with the files of the last commit of branch myBranch as well as the .git. The history only shows the last commit.
|
||||
|
||||
B) Updating the current working directory repository, moving it from its current state to a certain tag, calling for an aggressive garbage collection.
|
||||
./path/to/git_update.sh -r myTag -a https://git.mydomain/myname/myrepository.git
|
||||
Result: untracked files are still here. all the tracked files have been changed to respect the tag commit. The history only shows the last commit.
|
||||
git gc has been called with the --aggressive option, hence the objects in the .git have been repacked so as to optimize memory.
|
||||
CAREFUL: git branch still displays the original branch name and not the one of the tag.
|
||||
|
||||
|
||||
# Development process
|
||||
git_update.sh has been written by the French association Jean-Cloud (https://www.jean-cloud.net), in the process of developing Shlagernetes, an orchestration tool.
|
||||
Shlagernetes allows storing services on fallible second-hand servers and tries to consume the less possible resources.
|
||||
|
||||
Several tests have been performed in order to find the most energy-saving Git commands.
|
||||
You can find them under /test/performance_tests.
|
||||
Please refer to /doc/development_explanations for more information about the tests and choices made in the elaboration of git_update.sh.
|
||||
|
||||
# Testing
|
||||
The script test_git_update.sh can be found at /test/functional_tests. It will create a repository to test on before testing.
|
||||
Please refer to the readme of test and the readme of test/functional_tests.
|
||||
|
||||
# How to contribute
|
||||
If you have any suggestion or question about git_update.sh, please contact us at contact@jean-cloud.org.
|
||||
We would be delighted to hear your feedback. Feel free to send diff files with improvement suggestions.
|
||||
@ -1,34 +0,0 @@
|
||||
## Introduction
|
||||
test_git_update.sh performs functional tests on git_update.sh to verify that it is behaving as planned.
|
||||
|
||||
## Tests
|
||||
The script consists in nine tests. The three first ones test the cloning and the following test the updating of git_update.sh.
|
||||
|
||||
Each test consists of calling git_update.sh, eventually after some modifications, and compare the result with the expected result. The comparison is done by *_check functions, which modify each a boolean *_result.
|
||||
If all booleans of checks conducted are true, then the boolean case* is also true and the test is a success.
|
||||
|
||||
## Help extract
|
||||
NAME
|
||||
test_git_update.sh
|
||||
SYNOPSIS
|
||||
test_git_update.sh [-a] [-h] [-c] [-n number]
|
||||
OPTIONS
|
||||
-a excutes all the tests and cleans.
|
||||
-n number executes test number.
|
||||
-c cleans.
|
||||
-h prints the help.
|
||||
DESCRIPTION
|
||||
TEST0: git cloned in an empty directory
|
||||
TEST1: git cloned in an empty directory with tag
|
||||
TEST2: git cloned in an empty directory with branch
|
||||
TEST3: git updated fast-forward on main
|
||||
TEST4: git updated fast-forward on main with untracked file
|
||||
TEST5: git updated with underlying conflict on main
|
||||
TEST6: git updated, switching to another branch, deleting and adding files in the process
|
||||
TEST7: git updated, switching to a tag, deleting and adding files in the process
|
||||
TEST8: git updated, before and after changing a tag, deleting and adding files in the process
|
||||
TEST9: git updated fast-forward on submodule on main"
|
||||
|
||||
## Debugging
|
||||
In case of problem, it is adviced to run test_git_update.sh -c to clean the repository.
|
||||
This will force test_repo_creation.sh to run again and provide a new test repository.
|
||||
@ -70,4 +70,4 @@ DESCRIPTION
|
||||
```
|
||||
|
||||
## To go further
|
||||
To learn more about the process of testing different git methods conducted, please refer to doc/developement_explanations.md.
|
||||
To learn more about the process of testing different git methods conducted, please refer to [/doc/developement_explanations.md](/doc/developement_explanations.md).
|
||||
@ -1,68 +0,0 @@
|
||||
## Introduction
|
||||
Our objective is to find the least-consuming method in terms of memory and bandwidth resources.
|
||||
We are interested in cloning one specific state of the repository. We are not interested in its history or the possibility to change it from the server where it has been cloned.
|
||||
The tests rely on a repository created by test_repo_creation.sh. It measures memory and bandwidth usage for different git commande.
|
||||
|
||||
## Tests
|
||||
The script consists of twenty-nine tests (listed in the results below), based on three functions: generate_random_file, get_storage_used and get_bandwidth.
|
||||
|
||||
generate_random_file uses the bash command dd and /dev/random.
|
||||
get_storage_used uses the bash command du.
|
||||
get_bandwidth retrieves the output of Git commands and extracts the traffic displayed. This does not take submodule traffic into account.
|
||||
|
||||
The first five tests concern cloning.
|
||||
The following tests involve updating the repository using different commands, with three cases for each command: after adding a file, after deleting a file, after adding then deleting a file.
|
||||
|
||||
## Help extract
|
||||
NAME
|
||||
performance_tests.sh
|
||||
SYNOPSIS
|
||||
performance_tests.sh [-a] [-h] [-n number]
|
||||
OPTIONS
|
||||
-a executes all the tests.
|
||||
-n number executes test number.
|
||||
-c cleans.
|
||||
-h prints the help.
|
||||
DESCRIPTION
|
||||
This script allows you to measure memory and bandwidth usage. The first five test different cloning methods. Te following apply changes to the local remote before testing fetching and merging commands.
|
||||
TEST0: classic cloning
|
||||
TEST1: --single-branch cloning
|
||||
TEST2: --depth=1 --no-single-branch cloning
|
||||
TEST3: --depth=1 cloning
|
||||
TEST4: --depth=1 with reflog and gc cloning
|
||||
TEST5: sparse-checking 1M sample0 cloning
|
||||
_________________
|
||||
TEST6: classic fetching+checking out after addition of a 1M file
|
||||
TEST7: classic fetching+checking out after removal of a 1M file
|
||||
TEST8: classic fetching+checking out after addition then removal of a 1M file
|
||||
|
||||
TEST9: --depth=1 fetching+checking out after addition of a 1M file
|
||||
TEST10: --depth=1 fetching+checking out after removal of a 1M file
|
||||
TEST11: --depth=1 fetching+checking out after addition then removal of 1M a file
|
||||
|
||||
TEST12: --depth=1 fetching+checking out with reflog annd gc after addition of a 1M file
|
||||
TEST13: --depth=1 fetching+checking out with reflog annd gc after removal of a 1M file
|
||||
TEST14: --depth=1 fetching+checking out with reflog annd gc after addition then removal of a 1M file
|
||||
|
||||
TEST15: --depth=1 fetching+ --reset=hard after addition of a 1M file
|
||||
TEST16: --depth=1 fetching+ --reset=hard after removal of a 1M file
|
||||
TEST17: --depth=1 fetching+ --reset=hard after addition then removal of a 1M file
|
||||
|
||||
TEST18: --depth=1 fetching+ --reset=hard and reflog and gc after addition of a 1M file
|
||||
TEST19: --depth=1 fetching+ --reset=hard and reflog and gc after removal of a 1M file
|
||||
TEST20: --depth=1 fetching+ --reset=hard and reflog and gc after addition then removal of a 1M file
|
||||
|
||||
TEST21: --depth=1 fetching+checking out after addition of a 1M file in submodule
|
||||
TEST22: --depth=1 fetching+checking out after removal of a 1M file in submodule
|
||||
TEST23: --depth=1 fetching+checking out after addition then removal of a 1M file in submodule
|
||||
|
||||
TEST24: --depth=1 fetching+merging -X theirs with reflog and gc after addition of a 1M file
|
||||
TEST25: --depth=1 fetching+merging -X theirs with reflog and gc after removal of a 1M file
|
||||
TEST26: --depth=1 fetching+merging -X theirs with reflog and gc after addition then removal of a 1M file
|
||||
|
||||
TEST27: --depth=1 fetching+merging -s ours with reflog and gc after addition of a 1M file
|
||||
TEST28: --depth=1 fetching+merging -s ours with reflog and gc after removal of a 1M file
|
||||
TEST29: --depth=1 fetching+merging -s ours with reflog and gc after addition then removal of a 1M file"
|
||||
|
||||
## To go further
|
||||
To learn more about the process of testing different git methods conducted, please refer to doc/developement_explanations.md.
|
||||
51
test/readme
51
test/readme
@ -1,51 +0,0 @@
|
||||
# Introduction
|
||||
performance_tests contains tests who compare the memory and bandwidth usage of different Git commande in order to compare them.
|
||||
functional_tests contains tests who verify that git_update.sh is behaving as expected.
|
||||
Both rely on the same test repository that is created by test_repo_creation.sh.
|
||||
|
||||
# Creation of the test repository
|
||||
The script test_repo_creation.sh creates a remote/$REPO_NAME repository.
|
||||
## Help extract
|
||||
NAME
|
||||
test_repo_creation.sh
|
||||
SYNOPSIS
|
||||
test_repo_creation.sh [-h] [-l] [-s]
|
||||
DESCRIPTION
|
||||
This script creates a ./remote directory in the current directory, then creates a remote/$REPO_NAME git repository.
|
||||
This git repository is filled with randomly generated binary files described in the readme.md.
|
||||
OPTIONS
|
||||
-h prints the help.
|
||||
-l [link] adds the linked online repository as remote and pushes changes to it. Must be en empty repository.
|
||||
-s creates a submodule remote/$SUB_NAME and includes it in remote/$REPO_NAME.
|
||||
|
||||
# Description of the created repository
|
||||
Here is a history of the commits:
|
||||
### branch main
|
||||
commit sample0 1M created
|
||||
commit sample1 1M created
|
||||
commit sample3 5M created
|
||||
commit sample4 1M created
|
||||
commit sample3 5M deleted (tag : tagging_point)
|
||||
[if -s is selected] commit adding $SUB_NAME module
|
||||
### branch secondary
|
||||
commit sample0 1M created
|
||||
commit sample1 1M created
|
||||
commit sample2 500K created
|
||||
|
||||
Which gives the latest state:
|
||||
### branch main
|
||||
sample0 1M
|
||||
sample1 1M
|
||||
sample4 1M
|
||||
### branch secondary
|
||||
sample0 1M
|
||||
sample1 1M
|
||||
sample2 500K
|
||||
|
||||
If the -s option is selected, the submodule is created, and integrated in the last commit of main, with this history of commits:
|
||||
### branch main
|
||||
commit first 1M sample created
|
||||
|
||||
which gives the latest state:
|
||||
### branch main
|
||||
sub_sample0
|
||||
Loading…
Reference in New Issue
Block a user