int-72h / pytoast

A simple, low overhead content distribution system
BSD 3-Clause "New" or "Revised" License
8 stars 2 forks source link

dedicated revision information folder #18

Open Jan200101 opened 2 years ago

Jan200101 commented 2 years ago

Right now only one special file is used .revision to store the current revision ID.

But in the future more information might get added that would need to be stored in a known good location.

For this I propse a .tvn folder with .revision becoming .tvn/revision

for the sake of backwards compatibility .revision can be checked for if .tvn/revision is not present.

This would require coordinated changes to both toast and int-72h/oflauncher-rei which is why I am looking for feedback on this before creating Pull Requests

w3bb commented 2 years ago

Why not just have a .tvn file?

w3bb commented 2 years ago

Also how the launcher handles managing local revisions is not a part of TVN, it's an oflauncher specific thing (which other launchers aim for for the sake of compatability.)

Jan200101 commented 2 years ago

Why not just have a .tvn file?

the goal is to have a dedicated folder for tvn version contents so that things like #19 have a known good location without trashing the root directrory

Also how the launcher handles managing local revisions is not a part of TVN, it's an oflauncher specific thing (which other launchers aim for for the sake of compatability.)

Outside of https://github.com/int-72h/toast/blob/1cdafc37958b7828775f1500424764f887781680/tvn/__init__.py#L48-L54 yes

This is mainly a proposal for how the wider format should adapt, obviously a third party launcher can simply go ahead and do its own things, but I prefer there being one defined standard that launchers can share

w3bb commented 2 years ago

the goal is to have a dedicated folder for tvn version contents so that things like https://github.com/int-72h/toast/issues/19 have a known good location without trashing the root directrory

What makes a file less well-known than a directory?

int-72h commented 2 years ago

i'm going to weigh in on this. the exising .revision file is simple, and any additional data can easily be added by making it a json file. Though on the filesystem, in the future if there's additional data, having the .revision file unspoilt and any additional data kept neatly in other files in one directory would be nice. Honestly I don't know which approach is better.

w3bb commented 2 years ago

Maybe we could have something like: openfortress/69

Instead of saying "this is revision 69" we could have "this is revision 69 of openfortress."

Jan200101 commented 2 years ago

the goal is to have a dedicated folder for tvn version contents so that things like #19 have a known good location without trashing the root directrory

What makes a file less well-known than a directory?

its not but a file is just a file, you either have to adopt a format like json, yaml, ini, toml, etc. or create your own format that needs to be parsed.

with a directory multiple things can be stored without having a dependecy on some format or having to parse things

Maybe we could have something like: openfortress/69

Instead of saying "this is revision 69" we could have "this is revision 69 of openfortress."

This would go back to your suggestion in #19 and a server hosting an identifier for itself.

w3bb commented 2 years ago

but a file is just a file, you either have to adopt a format like json, yaml, ini, toml, etc. or create your own format that needs to be parsed.

with a directory multiple things can be stored without having a dependecy on some format or having to parse things

I don't think there is a difference. A directory also has keys (file names) and values (contents of files.) Whether you choose to represent it with JSON or a filesystem the exact same problems exist.

This would go back to your suggestion in https://github.com/int-72h/toast/issues/19 and a server hosting an identifier for itself.

I thought he responded in the other issue for some reason.

int-72h commented 2 years ago

but a file is just a file, you either have to adopt a format like json, yaml, ini, toml, etc. or create your own format that needs to be parsed.

with a directory multiple things can be stored without having a dependecy on some format or having to parse things

I don't think there is a difference. A directory also has keys (file names) and values (contents of files.) Whether you choose to represent it with JSON or a filesystem the exact same problems exist.

Personally I think that dealing with an FS is easier than dealing with json at a lower level but the opposite at a higher one

w3bb commented 2 years ago

but a file is just a file, you either have to adopt a format like json, yaml, ini, toml, etc. or create your own format that needs to be parsed.

with a directory multiple things can be stored without having a dependecy on some format or having to parse things

I don't think there is a difference. A directory also has keys (file names) and values (contents of files.) Whether you choose to represent it with JSON or a filesystem the exact same problems exist.

Personally I think that dealing with an FS is easier than dealing with json at a lower level but the opposite at a higher one

There are libraries for JSON, there aren't for a custom filesystem structure.

int-72h commented 2 years ago

but a file is just a file, you either have to adopt a format like json, yaml, ini, toml, etc. or create your own format that needs to be parsed.

with a directory multiple things can be stored without having a dependecy on some format or having to parse things

I don't think there is a difference. A directory also has keys (file names) and values (contents of files.) Whether you choose to represent it with JSON or a filesystem the exact same problems exist.

Personally I think that dealing with an FS is easier than dealing with json at a lower level but the opposite at a higher one

There are libraries for JSON, there aren't for a custom filesystem structure.

A custom filesystem structure wouldn't need a JSON library (as much since the files may themselves contain json) since you're just looking for files.

Jan200101 commented 2 years ago

There are libraries for JSON, there aren't for a custom filesystem structure.

There are libraries that deal with filesystem stuff. And even better: most programming languages ship with code for filesystem logic!

w3bb commented 2 years ago

filesystem logic != filesystem structure

Jan200101 commented 2 years ago

you don't really need any special code or library to deal with files being in certain places though languages with first class filesystem support like Go or Python have it even easier

w3bb commented 2 years ago

You don't understand what I'm talking about. I'm talking about dealing with stucture, and specifically mapping that structure to memory.

Jan200101 commented 2 years ago

you need to deal with filesystem structure regardless making STEAM/steamapps/sourcemods/open_fortress/.revision into STEAM/steamapps/sourcemods/open_fortress/.tvn/revision is hardly a deal breaker.

Since there is officially only one file right now .revision this would not change a lot to the user, but it would be helpful for future proofing the launcher and other independent launcher implementations by giving them their own workspace to do stuff in, think .git.

In a similar sense I believe that doing it like svn and git and using a content-addressable filesystem is the way to go because it removes a dependency on any file format parser that can interpret the files.

I get the argument that json could be used for this since its already quasi-required for parsing revision data (though the revision data is only really a simple subset of json that could easily be parsed with a hand-rolled function) but I think keeping the storage and parsing logic requirements separate would be beneficial by keeping it simple

w3bb commented 2 years ago

I have a solution that I think works better, and is worth having a part of the TVS spec. It's a .tvsignore file. It's a simple list of directories that TVS should ignore when processing revisions. The problem is that gameinfo.txt gets overwritten during verification, so it acts as a request for programs to not touch some file. This has a positive side effect that launchers can programatically add some directories to ignore. If you want clients to not touch some directory, say .ofqt then you can append it to the file.

Jan200101 commented 2 years ago

I am not opposed to a .tvnignore file, which would especially be useful for power users wishing to keep modified files beyond updates and verifications.

But I do not think its a good replacement for a standardized location where a launcher can put internal data that a user will generally not have to care about.