`convert-svnexternals` script does not handle `svn:externals` with date-time revisions

[andry81]

The convert-svnexternals script relies on subgit.com import, which skips svn:externals records with the date-time syntax as described in the issue of another tool: https://github.com/dirk-thomas/vcstool/issues/243

So the resulted .gitsvnextmodules file would not has revision = XXX fields at all.

[marcows]

This is not an issue of the convert-svnexternals script. You might ask the SubGit developers to support date-time revisions in svn:externals.

To make the resulting deficit an issue of this script, the title has to be something like: convert-svnexternals does not handle .gitsvnextmodules sections without revision

And that's what I have to support for my use cases because we have several projects using svn:externals without any revision information - either by accident or with good intention on tags or even on purpose on trunk...

So I plan to add some kind of revision guessing feature that could help you as well. But before that I have something different in the pipeline.

[andry81]

You might ask the SubGit developers to support date-time revisions in svn:externals.

I don't need it anymore. I decided to completely move to the git and use fork of the vcstool with the sparse checkout feature. The .gitsvnextmodules file I just left as a history. No need to convert svn:externals for all commits, especially if you have many external git repositories to map with (need to know all these hashes before map to them). Enough to convert only svn:externals for the last commit.

And that's what I have to support for my use cases because we have several projects using svn:externals without any revision information - either by accident or with good intention on tags or even on purpose on trunk...

Just move to the git and use vcstool with the sparse checkout. This is enough and much more powerful. Yes, there is no date-time revisions, but it does not need much and you still can point a branch HEAD as just like in the SVN.

[marcows]

No need to convert svn:externals for all commits, especially if you have many external git repositories to map with (need to know all these hashes before map to them).

That was fully scriptable using refs/svn/map created by SubGit.

Enough to convert only svn:externals for the last commit.

Not for us, we want to preserve the history.

Just move to the git and use vcstool with the sparse checkout.

How should sparse checkout help in organisation of libraries shared across different projects? We'll definitely not switch to a monorepo. Submodules have their drawbacks, but are bad-mouthed too quickly.

you still can point a branch HEAD as just like in the SVN.

We want to have reproducible states and builds. Pointing to some latest HEAD revision is a nightmare.

[andry81]

That was fully scriptable using refs/svn/map created by SubGit.

That's semi scriptable, you still can not automate creation of external git repositories. You have at least use, for example, GitHub CLI to start automate of creation repositories at GitHub, but it still pain in arse.

Not for us, we want to preserve the history.

This is main mistake. You must leave it as is and start in the GIT from the last SVN changeset. This is the best choice. To ease the movement, you can start to use vcstool directly in the SVN replacing all svn:externals and then after a while convert the rest to the git. No need to convert all at once.

How should sparse checkout help in organisation of libraries shared across different projects? We'll definitely not switch to a monorepo.
Submodules have their drawbacks, but are bad-mouthed too quickly.

The vcstool is not a suggestion to use a monorepo repository. The sparse checkout works in the git just like svn:externals in the svn. You can checkout a subdirectory from an external repository instead of entire external repository. The submodules in the git is just a basic minimal functionality to start use an external repository, it does not mean you must use .gitmodules or git submodule add. The vcstool and scripts like that is just a wrapper around git init and git checkout, and can use your own file format to store the external repository list.

Take a look on other implementations which use the submodules as a base:

• https://github.com/chronoxor/gil
• https://github.com/ingydotnet/git-subrepo

These are use the submodules, but customizes them for its own needs.

How should sparse checkout help in organisation of libraries shared across different projects?

You just allocate a git repository for each library and use the vcstool yaml file to enlist and checkout external repositories by the list. Or use already allocated somethere on the internet or in the local in one of these stores: svn, git, hg, an archive file. Then use vcs import < .externals to checkout by the list file - .externals (or name it just like you want). No need to convert all repositories at the same time, you can just convert some repositories and still use the externals between those vcs systems.

We want to have reproducible states and builds. Pointing to some latest HEAD revision is a nightmare.

The git submodules does this from the box as a core functionality.

[marcows]

That was fully scriptable using refs/svn/map created by SubGit.

That's semi scriptable, you still can not automate creation of external git repositories.

Yes, we didn't automate creation of the Git repositories, but I only referred to generation of the SVN-Git-mapping.

Not for us, we want to preserve the history.

This is main mistake. You must leave it as is and start in the GIT from the last SVN changeset.

The main mistake is to think that others have the same requirements as you.

Take a look on other implementations which use the submodules as a base:

• https://github.com/chronoxor/gil
• https://github.com/ingydotnet/git-subrepo

These are use the submodules, but customizes them for its own needs.

No, it doesn't seem they use submodules under the hood.

I had a look into alternatives before the decision for submodules, and for our use case nothing of what I found was it worth going further than reading the README. And so it seems to be with these examples. It may work for people, it simply depends on your demands. And all my colleagues would get into trouble using their GUI because submodules are supported, but custom scripts are not obviously.

No need to convert all repositories at the same time, you can just convert some repositories and still use the externals between those vcs systems.

We don't have the need to convert all external repositories at the same time with synced Git-SVN mirror. We can use these libs in both SVN and Git superprojects during migration phase.

We want to have reproducible states and builds. Pointing to some latest HEAD revision is a nightmare.

The git submodules does this from the box as a core functionality.

I know that, we use submodules, and I'm glad it isn't allowed to point to the latest HEAD with Git submodules as it was possible with svn:externals. But I quoted your "you still can point a branch HEAD as just like in the SVN." in my reply anyway.

[andry81]

Yes, we didn't automate creation of the Git repositories, but I only referred to generation of the SVN-Git-mapping.

But this is extremely not enough. No point to map SVN to GIT without automation of git repositories creation and complete movement to the GIT after all. You will suffer from supporting of this mapping garbage when it start to desync. Mirroring from SVN to GIT always been a pain. Better to dump it off.

The main mistake is to think that others have the same requirements as you.

The requirements are made by the people, so you can always change it.

No, it doesn't seem they use submodules under the hood.

As I said, these does not use the submodules in a usual way. But instead does replace git init and git checkout. But nevertheless internally all external directories there created as submodules.

No need to convert all repositories at the same time, you can just convert some repositories and still use the externals between those vcs systems.

We don't have the need to convert all external repositories at the same time with synced Git-SVN mirror. We can use these libs in both SVN and Git superprojects during migration phase.

This is what I meant, the vcstool does support both vcs for migration purposes. And you actually is detached from vcs internals to use vcs builtin externals (submodules in the git and svn:externals in the svn), because using a wrapper over multiple vcs.

I know that, we use submodules, and I'm glad it isn't allowed to point to the latest HEAD with Git submodules as it was possible with svn:externals.

I see nothing wrong with point to the HEAD. It is a kind of vcs side automation for a user. If a user want to always use the HEAD, why not? In the git it was not possible some time ago.

PS: You can extract the issue into the discussion.

[marcows]

Our setup works pretty well and we use SVN-Git mirroring since a few years without problems.

And it's "Git", not "GIT".

[andry81]

https://support.tmatesoft.com/t/subgit-import-does-not-handle-date-time-revisions-in-svn-externals-properties/2987