search

leezer3 / OpenBVE

OpenBVE- A free train simulator
http://www.openbve-project.net
278 stars 52 forks source link

OpenBVE perhaps return to Debian again...!? #330

Open ginga81 opened 5 years ago

ginga81 commented 5 years ago

Relative with #261.

Today, via Twitter, a Debian Developer henrich is conversation to me. https://twitter.com/henrich

He say that if you mails to Debian's WNPP, he becomes to the sponser. Please send WNPP to submit@bugs.debian.org.

If recoverly to debian, you(Mr.leeser3) can continue to registration to debian directly.

At https://www.clear-code.com/blog/2014/3/7.html, the WNPP's prototype mail foramat is written.

Additionally, he checked your OpenBVE's .deb, and he advice to me. He say that some files are still not ready for registration. For example, debian/copyright. The deficiency files should to attach based from bellow. http://snapshot.debian.org/package/openbve/1.4.0.10-3/

Additionally, perhaps you are not use official deb packaging tool. If you use official deb creating tool, it is so smart.

I will continue to contact to developper henrich. If you sent to email to submit@bugs.debian.org, please announce to me. I tweet to him.

leezer3 commented 5 years ago

Interesting!

I agree debian/copyright is missing, will add that back again. The last conversation I had with @directhex (Previous package maintainer) was over email about 2 years ago.

I've got no real experience with Debian packaging, and the current package is based upon a combination of the original scripts and some tweaking of my own. IIRC Debian would want the compatibility data separated into a second package.

She was relatively happy to chat, but nothing happened, and I think she had a lot of other stuff on her plate too.

ginga81 commented 5 years ago

fetch from the http://snapshot.debian.org/package/openbve/1.4.0.10-3/, some more deficiency files.

He said that he will be a sponsor if you prepares and emails the necessary documents and the appropriate deb file. He said to me that it is busy now to the next debian release, so he said to me that every few days please mention. So please email me after preparation.

directhex commented 5 years ago

The last active package maintainer in Debian was actually @sladen, I just had some oversight as part of my general control of .NET in Debian

sladen commented 5 years ago

@leezer3 @ginga81: would encourage working with the packaging as-is:

was a careful import of released .zip snapshots into revision control, until Michelle moved on.

Is there a clear new upstream, from which to make new releases?

leezer3 commented 5 years ago

This repository is the upstream these days. I don't believe there has been any other fork / continuation released with any working development or new features :)

The release branch and the point releases should provide the snapshots you're after I think.

The history should be contiguous from the last source Michelle released although I'd have to check that; the packaging and other build scripts have been later additions on the top, as well as a lot of (ongoing) reorganisation.

Essentially, I think what this needs is a clear list of points Debian would like us to address?

leezer3 commented 5 years ago

OK, so I've had a quick look back into the commits and the linked repo from @sladen

The commit tree for this repo starts here: https://github.com/leezer3/OpenBVE/commit/7416862f3009bf32ad87f407237b48f5872e61d3

I appear to have started from a base of Michelle's last released ZIP source (1.4.3.0), and done some work before importing this into the GIT repo. This contained the full set of developer tools, data etc. After that, the history is contiguous.

Sladen has a set of repositories which are split by program, so that each of the tools lives in it's own repository. I'm not necessarily sure this is the 'correct' method of doing things in a game of this nature, especially as there is a massive amount of commonality between the different tools, and none of them have any real use outside the game. (N.B. One of the ongoing jobs is merging the duplicate code between the tools), either way it's two different / disparate approaches. Either way, Debian appears never to have distributed the tools, so I'm not sure how relevant this is.

Thoughts / TODO List:

sladen commented 5 years ago

@leezer3, it's 10 years ago, so things may be a little hazy.

For the content side, we had three examples (all thanks to Anthony):

The route/train/cab/plugins are not OpenBVE-specific per-se, they are in BVE-CSV fileformat and could also be used with eg. the original BVE Trainsim non-free freeware program + tools. Hence the bve- prefix.

It would be great to bring the tools into the main repository, and I'm sure Debian/Ubuntu would be very happy to follow whatever Upstream does! ;-)

leezer3 commented 5 years ago

No trouble.

I've been in this community since 1997 or there abouts, and have seen the entire gamut of BVE..... Started developing this rather by accident, but now things have rather started to get a lot more complex!

Main Program:

I've actually done some playing with cross-architecture proxying for i386 plugins: https://github.com/leezer3/OpenBVE/pull/230

This works acceptably(ish), but it's not code I was ever happy with. Being honest, it's something which needs someone with considerably more understanding of cross-architecture would need to look into properly again. We're also still limited to openGL 1.1 & Display Lists, which somewhat rules out most non x86 architectures.

Data:

The data included in the current releases has grown somewhat since the original days. At present, it's this:

At 5mb compressed, whether a separate package is worth the trouble is something Debian would have to decide. Distributing a deb from our site, I made the decision that supplying a single file was just easier, although obviously being in a repo means that it just gets pulled in as a dependancy :)

Tools:

My instinct would be to include with the main program, and use executable names prefixed with openBVE as follows:

openbve-objectviewer
openbve-routeviewer
openbve-traineditor
openbve-carxmlconvertor

etc.

An alternative would be to add a Developer Tools button / dialog to the main menu, which would avoid adding anything to the path?

Content:

I'm happy to provide 3 more fully cross-platform locos & assorted consists ( GWR 81xx, GWR Manor, BR Class 52 ) in public domain / BSD-3 if we feel that is a good idea. None of these work correctly on the original BVE trainsim however. (n.b. openBVE introduced both the 3D cab and train exteriors, so neither of these work per-se with the original BVE Trainsim)

Route-wise is somewhat more complex, and I don't think there's anything easily available that could be added.

As opposed to Michelle's original managed content system. a relatively early implementation was a basic package manager. This isn't anything particularly special, just reads a metadata XML from a zip, and allows installation / removal etc. I don't know quite how this will relate to the Debian content packages, but it was considered important to have a format which automatically packaged / installed content, and was cross-platform.

Other:

The issue of the licence mess Michelle left us with pops up every once in a while: https://github.com/leezer3/OpenBVE/issues/305

We have agreement from all the non-trivial contributors to this repository that a move to BSD is acceptable in principle, and TBQH the current licence probably would allow us to do this anyways. I'd be interested if you have comments or preferences in this regards.

leezer3 commented 5 years ago

OK, have done rather a lot of messing around with the generated deb.

This is now clean of all red lintian warnings, other than the lack of a changelog. I haven't currently got a good way to pipe this into the requisite file, so it would probably have to be created manually by whoever builds the debian package.

However, there are still some flaws:

This really needs someone familiar with Debian packaging to take a look now though. I can read Lintian warnings just fine, but I don't know enough about the fine details of what's happening.

directhex commented 5 years ago

Okay, so, taking a look at the packaging metadata, I feel it would be much easier to maintain in the longer term by using some newer conventions - right now it's using format version 5 from 2005 (https://github.com/sladen/openbve/blob/debian/debian/compat) and 75% of that file would go away with version 7 from 2008. https://salsa.debian.org/dotnet-team/xsddiagram/blob/master/debian/rules is an example of a Debhelper 7 rules file for a project built entirely from sln/csproj - and half of that file is a custom rule for downloading the .tar.gz.

Some of the automagic things in dh7 would likely help you

directhex commented 5 years ago

Sorry, I should have given a bit more context as to what's happening.

%:
    dh $@ --with cli

Means "for every Debian packaging rule, just do whatever the dh command wants to do, but add the .NET-specific commands defined in /usr/share/perl5/Debian/Debhelper/Sequence/cli.pm"

Then for every command dh wants to shell out to, you can choose to override it by adding a override_dh_foo: target, e.g. override_dh_auto_build (which detects things like autotools or cmake or meson) to instead call xbuild on the sln. You'd throw away everything in the existing rules and leave something like:

#!/usr/bin/make -f
export DH_VERBOSE=15
BUILDDIRS = openBVE/OpenBve openBVE/OpenBveApi \
 openBVE/OpenBveAts \
 openBVE/Sound.Flac openBVE/Sound.RiffWave \
 openBVE/Texture.Ace openBVE/Texture.BmpGifJpegPngTiff
TARGET = $(CURDIR)/debian/openbve
EXEC_TARGET = $(TARGET)/usr/lib/openbve
EXEC_PLUGIN_TARGET = $(EXEC_TARGET)/Plugins
#DEBUG_CONFIGURATION=Debug
DEBUG_CONFIGURATION=Release

override_dh_auto_build:
    for builddir in $(BUILDDIRS); do \
      (cd $$builddir && xbuild /p:Configuration=$(DEBUG_CONFIGURATION) *.csproj) || exit 1; \
    done

override_dh_auto_clean:
    for builddir in $(BUILDDIRS); do \
      (cd $$builddir && xbuild /t:Clean *.csproj); \
      rm -rf $$builddir/bin $$builddir/obj; \
    done

override_dh_auto_install
    #do most of this stuff in debian/install, maybe with dh-exec
    lynx -dump -nolist $(CURDIR)/debian/changelog.html > $(TARGET)/usr/share/doc/openbve/changelog

override_dh_clideps:
    dh_clideps -d -r --exclude-moduleref=AtsPluginProxy.dll

%:
    dh $@ --with cli
leezer3 commented 5 years ago

Thanks!

The current in-tree script we've got (and which I've been modifiying today) actually builds the solution (Uses mcs , not xbuild) before copying it into the Debian package structure and calling dpkg-deb to build the thing. IIRC the only thing that got used from the original package was a stripped down metadata file so that we'd install over an existing install gracefully.

After today's work, it declares format V7, which if I understand correctly isn't actually doing much in this case, as I'm essentially just packing binary files.

If I understand what's going on with the rules file correctly, the only real advantage would be that we could just call the dpkg creation command in the root directory, and dpkg would sort everything else out? Not entirely clear whether it's a Debian requirement for things to work in this way, or just whether the deb must conform to standards?

directhex commented 5 years ago

For a "use Debian packaging on a tarball of binaries not source" your example is https://github.com/mono/linux-packaging-nuget

But bear in mind this strategy isn't ok for inclusion in packaging repositories

leezer3 commented 5 years ago

OK, so I've got some progress, but I'll admit this is giving me a real headache.

Due to a lot of changes to the structure of things (specifically intended to reduce the amount of duplicate code between the main program and the viewers), the build order is rather important. Attempting to do this manually is a PITA and requires manual work every time something changes.

As it looks likely that the tools may be included with the main program, I think this makes the easiest method of doing things to be to simply build the main solution file directly, and let this handle the build order itself.

I think we'll need a separate target, or a call to delete the data, but keep that back until the deb is actually generating.....

Current state of play (n.b. Please ignore the version number etc, I just needed something to stuff in there to keep the scripts happy) https://github.com/leezer3/OpenBVE/commit/afd9ea90bcaf937d73fc7a3024314a45089422d9

This is the current test commit. At the minute, dpkg-buildpkg is failing with the following (truncated for brevity) output:

    Could not read /mnt/downloads/SourceCode/debian/openbve.substvars
    Could not read /mnt/downloads/SourceCode/debian/openbve.substvars
    Could not read /mnt/downloads/SourceCode/debian/openbve.substvars
    (grep -a -s -v cli:Depends debian/openbve.substvars; echo "cli:Depends=libc6 (>= 2.24) | libc6.1 (>= 2.24) | libc0.1 (>= 2.24), libgl1-mesa-glx, libmono-corlib4.5-cil (>= 4.6.1.3), libmono-system-core4.0-cil (>= 4.6.1.3), libmono-system-drawing4.0-cil (>= 4.6.1.3), libmono-system-windows-forms4.0-cil (>= 1.0), libmono-system-xml-linq4.0-cil (>= 4.2.0), libmono-system-xml4.0-cil (>= 4.6.1.3), libmono-system4.0-cil (>= 4.6.1.3), libopenal1, libx11-6 (>= 2:1.6.0), libxcursor1 (>> 1.1.2), libxi6 (>= 2:1.6.99.1), libxinerama1, libxrandr2 (>= 2:1.5)") > debian/openbve.substvars.new
    mv debian/openbve.substvars.new debian/openbve.substvars
    grep -a -s -v '^cli:Recommends=' debian/openbve.substvars > debian/openbve.substvars.new || true
    mv debian/openbve.substvars.new debian/openbve.substvars
    grep -a -s -v '^cli:Suggests=' debian/openbve.substvars > debian/openbve.substvars.new || true
    mv debian/openbve.substvars.new debian/openbve.substvars
dh_clideps: Error: unresolvable module references or missing shlibs entries, please check above errors!
debian/rules:20: recipe for target 'override_dh_clideps' failed
make[1]: *** [override_dh_clideps] Error 255
make[1]: Leaving directory '/mnt/downloads/SourceCode'
debian/rules:23: recipe for target 'binary' failed
make: *** [binary] Error 2
dpkg-buildpackage: error: fakeroot debian/rules binary gave error exit status 2
directhex commented 5 years ago

Can you stick the full build log in gist? Some is missing

Also, it looks like you included some temporary files in your commit - you can delete all files with substvars or debhelper in the filename

leezer3 commented 5 years ago

https://gist.github.com/leezer3/bf8a4e5fd3ab1485cb35b209cb6d3650

Full build log.

I'm sure there are plenty of other flaws in my build logic, so please don't spend too much time on this. Happy enough to keep fiddling myself once I understand it a little better :)

directhex commented 5 years ago

dllmap issues - the installed assemblies dllimport native libs whose packages can't be determined - e.g. ole32.dll (unsurprising, since that's Windows only)

You can either exclude the named things by adding --exclude-moduleref calls to the override_dh_clideps target in debian/rules, or add them to a dllmap , or ideally a combination of both. i.e. dllmap for libXxf86vm -> libXxf86vm.so.1, exclude for /System/Library/Frameworks/ApplicationServices.framework/Versions/Current/ApplicationServices

You also have one shlibs issue for libSDL2-2.0.so.0 - change the Depends: line in debian/control to Depends: ${cli:Depends}, ${shlibs:Depends}, ${misc:Depends} for full automatic dependency resolution, including a fix for the missing libSDL2 dependency

leezer3 commented 5 years ago

Thanks.

I've got it building now, although I had to disable EGL, KVM and some other stuff from checking. I don't think that matters, as we don't use these features, they're just present in the openTK dll we import?

TODO:

leezer3 commented 5 years ago

openbve_1.6.0.0-1_all.zip

Test generated deb & the two other generated files. Note that it's unsigned, and the version number is temporary.

ginga81 commented 5 years ago

・Who should be packaging / signing? I think that this is may be yourself(Mr.leeser3).

・Related, we need to figure out who the maintainer actually is. We have an offer from henrich, @sladen is still I think the current maintainer, or what? I think that if the build is completely new, someone who candidates choose our project. I think may be that we cannot choose the new maintainer directly. Mr. henrich is only become the sponser of WNPP, not a maintainer.

ginga81 commented 5 years ago

To create deb package more easily, may be helps this tool? https://lintian.debian.org/ This tool is written here. https://www.clear-code.com/blog/2014/4/3.html

sladen commented 5 years ago

For the record, the Maintainer: is a shared team:

ginga81 commented 5 years ago

Mr.leeser3, if you are ready to deb package, please send the email to submit@bugs.debian.org with WNPP template, and report me. I tweet to Mr.henrich.

leezer3 commented 5 years ago

I'm still working on it in the linked branch :)

We now have basic wrappers for the Object Viewer / Route Viewer binaries, but for some reason the menu entry isn't triggering correctly. Other than that, the data needs sorting and packaging correctly too (assuming that is Debian want to retain the two package structure)

When I'm done, the linked branch will get merged into mainline, and hopefully added to the CI. At that point, we should be ready to release the 'real' 1.6.0.0, which is what will want submitting to Debian :)

ginga81 commented 5 years ago

I heard that this marge is ready to come back to debian from @s520 . Are you ready for WNPP and send to submit@bugs.debian.org? If so, I will tweet to henrich.

leezer3 commented 5 years ago

It seems to build the deb OK on Stretch, but not on Jessie. Haven't had a chance to investigate yet.....

s520 commented 5 years ago

Jessie's support is LTS, so I don't think we need to worry too much. As you know, LTS only makes security updates. So I think the problem is whether there is a problem with Buster.

leezer3 commented 5 years ago

Jessie wasn't building due to a lack of SDL2, so that's minor :)

RFP bug now filed here: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=927278

I think the next step is for me to register on Debian mentors, & see if I can figure out how to get the built deb up there: https://mentors.debian.net/

ginga81 commented 5 years ago

I tweeted to Mr.henrich to get in touch with. Please wait.

ginga81 commented 5 years ago

From Mr.henrich, he said that he will check this to be a sponsor at least weekend. Please wait.

sladen commented 5 years ago

See also:

Hopefully the data packages can hang around long enough that they won't need re-uploading again.

leezer3 commented 5 years ago

In many ways I'd prefer for there to be one single package. Makes maintenace so much easier :)

The demo route / train definitely need to be separate though.

When push comes to shove, it depends on what Debian's requirements are likely to be. Probably needs discussion on tge WNNP bug.

~8mb total is insignificant these days, and the language files change rapidly enough with features that they really ought to be packaged with the main build.

sladen commented 5 years ago

Would encourage walking before running.

ie. Replace the executable .deb and get everything working again as before. Then consider moving things around.

leezer3 commented 5 years ago

The trouble with that is that they're intertwined.

The current version of openbve-data isn't going to help us much:

TLDR: If keeping the two package structure, a new version of openbve-data will be required in order for things to work. Whilst I'm not sure off the top of my head what would happen if we attempted to use the data files in openbve-data instead of the current stuff, a lot of features would definitely not work, and I suspect it'd plain crash.

sladen commented 5 years ago

That is what Depends: openbve-data (>= 1.4.0.0) is for; the Depends: is updated whenever something fundamentally changes and the program requires updated data to start without crashing.

ginga81 commented 5 years ago

openbve-data package is already to 'standalone install package', isn't now?

Because, the 'main program has purged'.

I suggest that only to remove openbve-data package from the recommended package list and announce 'this package has been merged by update, so do not need to use openbve-data package'. After 'recovery', I suggest the openbve-data package remove from debian. I think that these process are more smartly.

Our project must be 'recovery' first.

I found that the additionaly issue. bve-route-cross-city-south bve-train-br-class-323 bve-train-br-class-323-3dcab These packages are also has been removed. These packages are very important to play the firsttime and to check run normally. And everyone can know 'What is OpenBVE'.

We are attempt to create for bundle in main program's demo-route and train. But unfortunately, not to ready now.

I think that these packages are also must to recovery.

ginga81 commented 5 years ago

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=927278#8 Mr.henrich review and reported about this. Please check and change, and re-upload.

ginga81 commented 5 years ago

Such as our project case of removed from debian, to do not happen remove again future, Mr.henrich wrote this BTS. Please check it, and if you have some opinion, please write reply to this BTS. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=927776

s520 commented 5 years ago

@leezer3 We asked Mr.henrich to point out some problems with the package. Next, we want you to correct the points pointed out by Mr.henrich. What is the current situation?

leezer3 commented 5 years ago

I replied to Henrich's last message in the bug, haven't heard anything else yet.

Was hoping that there would be a little more clarification on some points, and it's not quite clear how exactly he wants the changes; He was suggesting that the Debian packaging stuff be done in the Salsa repo, not this one.

It's on my list of things to try and have a proper look at this weekend, but things are in progress :)

v1993 commented 5 years ago

Is there any progress on this?

leezer3 commented 5 years ago

Not really at this end.

It's been passed to Henrich, and I don't really have a lot of control over what's going on. He suggested some changes, but also was considering doing things himself- I did ask for clarification on exactly what he wanted, but haven't had anything back,

Either way, people who aren't developers here are busy, don't bug them too hard :) It'll get sorted eventually I hope.

ginga81 commented 5 years ago

Mr.henrich (link: https://wiki.debian.org/HidekiYamane) wiki.debian.org/HidekiYamane and we are stay in touch with him. But in the position, he is working to release the Debian 10 currently. In this situation, he is difficult to check our bts. At June 7 2019, we asked to him, and he answerd to us that 'I'm not forgetting, I'm understanding'. We think perhaps that after the release of debian 10, he checks bts again for us.

ginga81 commented 5 years ago

Debian10 buster released, but Mr.henrich have to speech at debconf19. So, he went to debconf19, and he is writing speech manuscript of debconf now. After the debconf and at least August, if he is not busy again, I think may be he can continue again.

ginga81 commented 5 years ago

Recently, we have contacted with @henrich. He said to us that is... 1.An developer authority for the salsa repository has been granted to leeser3, so you can be dealt with. 2.At all souce code, check and delete the texts. The guide line is showing bellow link. https://debian.org/doc/packaging-manuals/copyright-format/1.0/ For example, at the Assimp perser's souce code, it is written with the original contents such as 'please read these sample DirectX data'. But, we do not using with propraietary codes and datas at all. These texts are perhaps to missreading when someone who reads the source code. So, he said that remove these contents.

And, I want to talk with not this, I want to talk with Email. I sent the Email to you, please read it.

sladen commented 5 years ago

@ginga81: please can the fix-list from the email be posted directly?

(The older openbve packages in Debian did have complete metadata already).

s520 commented 5 years ago

The Debian maintainer said that there was no problem even though the copyright notice was ambiguous at the time, but it is now strict. Is this a mistake?

And the maintainer didn't say anything more than that. He asked us to do the work first.

ginga81 commented 5 years ago

@sladen Oh, I am sorry. This Email is not the list. This is my private Email, and another topic. Email contact is a bit long time, for example three or more days. So I wrote.

sladen commented 5 years ago

u/s520: ginga81: please make sure all information is copied to the Github issue tracker, or the Debian Debbugs bug tracker.

Commends like "X said that Y said that I think Z said" are vague.

To be able to resolve/fix a bug, we need exact wording about exactly what is broken/wrong/needs fixing.

s520 commented 5 years ago

I am not my native language. This exchange took place on Twitter in Japanese. This information is a solid statement.