Difference between revisions of "Fossil"
From Dan Shearer CV
| Line 1: | Line 1: | ||
| − | [https://www.fossil-scm.org/ Fossil] source code management system is the only realistic alternative to Git, and has had 15 years of development and testing. I |
+ | [https://www.fossil-scm.org/ Fossil] source code management system is the only realistic alternative to Git, and has had 15 years of development and testing. I now use Fossil. |
| + | == Why Not Git or GitHub? == |
||
| ⚫ | |||
| + | |||
| + | I don't think Git is bad, and GitHub was a major advance on SourceForge in the same way that SourceForge was a major advance on plain CVS. All my Fossil code is mirrored on GitHub, and it is a good place to search for existing projects. |
||
| + | |||
| + | However I needed to find an alternative to Git and GitHub because: |
||
| + | |||
| + | * GitHub seems focussed on maximising the number of total projects which use their proprietary processing pipelines. For GitHub, every cloned project is a financial win, but isn't clear to me that is a win for the world. |
||
| + | * I could not convince GitHub to fix visual accessibility problems, and I had multiple team members with visual impairments. GitHub have billions in the bank, and I spoke to several very polite managers at length, but GitHub still refuse to implement years-old W3C accessibility standards. That's not acceptable. |
||
| + | * Even if you only ever use a git commandline, Git comes with a lot of pain... software developers much more skilled than me wrestle with Git and its complexities. It doesn't feel right to me that a development team should need to worry about losing work, or use an interface so complex that [https://git-man-page-generator.lokaltog.net/ paradoy man pages look real!]. |
||
| + | |||
| + | Then there are security issues. [[LumoSQL]] and [[Sweet Lies]] are EU-domiciled open source projects which focus on security, and: |
||
| + | # GitHub has [[Analysis of EU-US Privacy Shield|US Cloud issues]], and so should not be used in the EU |
||
| + | # Git encourages users to break the Merkle tree. Rather than an inviolate historical record, it's a curated version (especially with 'rebase') |
||
| + | # It is difficult to find the descendants of check-ins in Git. It is so difficult that neither native Git nor GitHub provide this capability, and you need to write code to crawl the commit log. This makes it hard to find what descendent code may have been affected by an upstream bug. |
||
| + | |||
| + | The Fossil website has a pretty balanced [https://fossil-scm.org/home/doc/trunk/www/fossil-v-git.wiki comparison of Fossil v Git] that is much more comprehensive than what I have written here. |
||
| + | |||
| ⚫ | |||
| + | |||
| + | |||
| + | == Work Done on Fossil == |
||
| + | |||
| + | Before I could use Fossil, I needed some changes: |
||
| + | |||
| + | * Fossil was not then a commodity, off-the-shelf SCM, and I needed users to be able to just get it easily for their favourite operating system. |
||
| + | * Fossil only had one implementation. That made me uncomfortable with Git too. Why would we have a vital standardised data format with only one set of very complex tools that can read it? |
||
| + | |||
| ⚫ | |||
| ⚫ | |||
| ⚫ | |||
* I became a [https://fossil-scm.org/forum/forumpost/daf8242f38b574c6 temporary packaging intermediary] with the main distributions. This has been successful... recent operating systems all carry recent versions of Fossil, and this now appears to be self-sustaining. There was a lot of private community interaction to make this happen. |
* I became a [https://fossil-scm.org/forum/forumpost/daf8242f38b574c6 temporary packaging intermediary] with the main distributions. This has been successful... recent operating systems all carry recent versions of Fossil, and this now appears to be self-sustaining. There was a lot of private community interaction to make this happen. |
||
| − | * I somewhat assisted [https://fossil-scm.org/forum/forumpost/ec5155da3c822f13342bc7 Stephan Beal's libfossil] to roar back into life as a second, completely independent implementation of the Fossil data model. Multiple implementations are really important |
+ | * I somewhat assisted [https://fossil-scm.org/forum/forumpost/ec5155da3c822f13342bc7 Stephan Beal's libfossil] to roar back into life as a second, completely independent implementation of the Fossil data model. Multiple implementations are really important and being a library means the world can have multiple front-ends alternatives to the official Fossil app. I don't want my projects locked into Fossil any more than Github, although I am perfectly happy with Fossil for now. libfossil is great insurance. |
* I completed a privacy review of Fossil, and [https://fossil-scm.org/forum/forumpost/d4d4eff808166ed57bb8a81062807267baa1572f5d41247af4baa73ce69afb95 debated my proposal] in public. Some of that involved [https://fossil-scm.org/forum/forumpost/d4e051ac794fce31f232f6339fd445 discussion of privacy arcanae]. |
* I completed a privacy review of Fossil, and [https://fossil-scm.org/forum/forumpost/d4d4eff808166ed57bb8a81062807267baa1572f5d41247af4baa73ce69afb95 debated my proposal] in public. Some of that involved [https://fossil-scm.org/forum/forumpost/d4e051ac794fce31f232f6339fd445 discussion of privacy arcanae]. |
||
| ⚫ | |||
| − | * I have had discussions with CyberSecurity researchers in two universities about the EU Privacy Shield issue facing Github as a US cloud, and what best to replace Github with in Europe. This discussion is ongoing. |
||
| ⚫ | |||
| + | |||
| + | == Fossil as a LumoSQL Test Case == |
||
| − | Not only is Fossil a better SCM for |
+ | Not only is Fossil a better SCM for the needs of my projects, but it is also a very demanding test case for LumoSQL. Fossil is built on SQLite, in fact Fossil and SQLite are symbiotic projects, and Fossil is the one SQLite application all SQLite developers are guaranteed to use. If Fossil can run on LumoSQL without a problem, and potentially even with some advantages, then it will have passed a major milestone. |
| − | It may sound strange to some to use anything other than Git, which is thoroughly discussed at this [https://fossil-scm.org/home/doc/trunk/www/fossil-v-git.wiki comparison document] I have contributed to. For the record though, here are some of my specific reasons: |
||
| ⚫ | |||
| − | # '''Git focusses on ancestors, not descendants'''. It is possible to find ancestor descendant commits by parsing git logs, but git does not help you do this even though it is a very important feature for checking security issues. LumoSQL is itself a modification of a massively forked-and-vendored codebase, and since LumoSQL combines this with many other codebases, locating descendants is important. An SCM is for managing a Directed Acyclic Graph (DAG) of checkins, and a DAG can be traversed in any direction. So this counts against Git. |
||
| − | # '''Git provides widely-used features to change history'''. One of the perceived benefits to using Git is that it encourages pull requests from people who have previously cloned your tree. It does not seem to be good design to accept clean trees from committers, because that tree has a reason for being the way it is. Git is like Toad of Toad Hall in the sense that it is about what we wished had happened in an ideal world. A security project would like to know what ''really'' happened, not what we ''wanted to have happened''. It isn't the right mindset. |
||
| − | # '''All Gits Lead to Github'''. Github is currently the best index of open projects and code. If we run our own Git server we will still mirror to Github (or perhaps GitLab). And then that is where the issues will be raised, and pull requests made, and the authentication namespace, because while we are compatible with the Git protocol we are realistically not going to try to duplicate all the rest of the infrastructure. And Github has many problems. Not least that it is extraordinarily inaccessible to people with quite common eyesight problems, and I have spent a lot of time asking Github to fix these problems in vain. |
||
| − | # Git was not intended for ordinary projects. Git provides an excellent solution for the Linux kernel, and, after some scaling-up, the even more enormous internet Microsoft code repository for all their products. But these contain tens and hundreds of millions of lines of code. Only a small handful of projects are anything like that big... for comparison, the major open source databases are around two million SLOC each (and SQLite is a slim 250 thousand SLOC.) Git is not sized correctly for most projects, or focussed on providing more certainty for developers of ordinary-sized projects with less than a million or so lines of code. |
||
[[Category:Software Development]] |
[[Category:Software Development]] |
||
Revision as of 06:32, 26 November 2021
Fossil source code management system is the only realistic alternative to Git, and has had 15 years of development and testing. I now use Fossil.
Why Not Git or GitHub?
I don't think Git is bad, and GitHub was a major advance on SourceForge in the same way that SourceForge was a major advance on plain CVS. All my Fossil code is mirrored on GitHub, and it is a good place to search for existing projects.
However I needed to find an alternative to Git and GitHub because:
- GitHub seems focussed on maximising the number of total projects which use their proprietary processing pipelines. For GitHub, every cloned project is a financial win, but isn't clear to me that is a win for the world.
- I could not convince GitHub to fix visual accessibility problems, and I had multiple team members with visual impairments. GitHub have billions in the bank, and I spoke to several very polite managers at length, but GitHub still refuse to implement years-old W3C accessibility standards. That's not acceptable.
- Even if you only ever use a git commandline, Git comes with a lot of pain... software developers much more skilled than me wrestle with Git and its complexities. It doesn't feel right to me that a development team should need to worry about losing work, or use an interface so complex that paradoy man pages look real!.
Then there are security issues. LumoSQL and Sweet Lies are EU-domiciled open source projects which focus on security, and:
- GitHub has US Cloud issues, and so should not be used in the EU
- Git encourages users to break the Merkle tree. Rather than an inviolate historical record, it's a curated version (especially with 'rebase')
- It is difficult to find the descendants of check-ins in Git. It is so difficult that neither native Git nor GitHub provide this capability, and you need to write code to crawl the commit log. This makes it hard to find what descendent code may have been affected by an upstream bug.
The Fossil website has a pretty balanced comparison of Fossil v Git that is much more comprehensive than what I have written here.
For me, it matters that Fossil is an append-only, non-repudiable Merkle tree with strong crytographic guarantees. Git is not.
Work Done on Fossil
Before I could use Fossil, I needed some changes:
- Fossil was not then a commodity, off-the-shelf SCM, and I needed users to be able to just get it easily for their favourite operating system.
- Fossil only had one implementation. That made me uncomfortable with Git too. Why would we have a vital standardised data format with only one set of very complex tools that can read it?
So I invested significantly in Fossil, and these problems were fixed:
- I became a temporary packaging intermediary with the main distributions. This has been successful... recent operating systems all carry recent versions of Fossil, and this now appears to be self-sustaining. There was a lot of private community interaction to make this happen.
- I somewhat assisted Stephan Beal's libfossil to roar back into life as a second, completely independent implementation of the Fossil data model. Multiple implementations are really important and being a library means the world can have multiple front-ends alternatives to the official Fossil app. I don't want my projects locked into Fossil any more than Github, although I am perfectly happy with Fossil for now. libfossil is great insurance.
- I completed a privacy review of Fossil, and debated my proposal in public. Some of that involved discussion of privacy arcanae.
- After being accepted as a code contributor, I have made 31 commits to the Fossil tree so far
- I have made over 150 forum postings
Fossil as a LumoSQL Test Case
Not only is Fossil a better SCM for the needs of my projects, but it is also a very demanding test case for LumoSQL. Fossil is built on SQLite, in fact Fossil and SQLite are symbiotic projects, and Fossil is the one SQLite application all SQLite developers are guaranteed to use. If Fossil can run on LumoSQL without a problem, and potentially even with some advantages, then it will have passed a major milestone.