search

wrye-bash / wrye-bash

A swiss army knife for modding Bethesda games.
https://wrye-bash.github.io
GNU General Public License v3.0
476 stars 81 forks source link

Beta releases #260

Closed Utumno closed 2 years ago

Utumno commented 8 years ago

We need to come up with a versioning scheme and release procedure for betas. Once done we need to add a wiki page and rewrite second posts and nexus (minimal rewrite, I can do that). I published a gazillion unofficial betas for 306 - I was just bumping bass version a la 306.201512021746 and packing that - I would like to be able to continue doing so off my wip branch. Very nifty as I knew exactly which version people were using, and I could publish as often as I needed people to test breaking (or fixing) changes. On the other hand we need to publish official 307 betas in nexus. We could tag the release and do nothing with the version in bass but then we will receive bugdumps saying just 307 which is not a great idea, trust me on that. We could branch off, bump and tag and then delete the branch and leave the tag up for some time - not sure how that plays with github (or git for that matter) - but it will allow us to have arbitrary version numbers off dev, while not touching bass. This issue has a TODO character also - as whatever versioning scheme we come up with we must make sure it is supported by the code.

So what you say ? What others are doing here in github ? Any gotchas I can't think of ? Any links to release procedure docs ?

I want to have my branch merged, and the skyrim patchers merged and then we are ready for a beta - but this will take couple weeks or more. Still better decide on those things now.

Metallicow commented 8 years ago

I'd go with the international ISO 8601 Year.Month.Date format https://en.wikipedia.org/wiki/ISO_8601 subnumbers could be attached for beta's Ex: .(Hour.Minute.Second) It works very nicely(for the cpu), besides a users personal prefs and is also "quote" standard...

lojack5 commented 8 years ago

The only thing I don't like about majorversion.year.month.day is that it clashes with versioning of non-beta releases. That is, we'd be doing something along the lines of majorversion.minorversion.possibly more. Now it'll be easy to distinguish between the two for a human, but if we ever need code that can tell them apart that could get tricky.

I'd like something more along the lines of: non-beta: Major.Minor.last two can be anything beta: Major.Minor.Year.Month&Day

Advantages:

Note: Keep in mind we're constrained a little by the Windows version fields - 4 ints of size WORD.

Anyway, my 2 cents.

Utumno commented 8 years ago

No better way to find out than rolling out a beta - it's about time. From my side I want in:

Those I can do in a week to ten days. Run in a series of bugs I kept fixing in the last few dev commits - and caused some regression when hurriedly implemented #53 - patience.

Another thing that's in for the beta, ofc:

@Sharlikran I need your help here. Now is the time, I'm on a holiday. Can you do the rebase and squashing ?

@lojack5: will you be around ? Having an eye on what's going on on dev is already help enough but I could use more. Re: beta and tagging - maybe merge to master and tag there as for normal releases ? We are going for a 307-beta1 thing (not sure if supported by the packaging script) or a 306.1 - what 'd you say ? There must be some minimal work done on meta too to support fallout 4 etc - maybe consider dropping posts per forum, too much overhead. Fallout 4 the patchers and the few enhancements/fixes (and performance fixes) I will throw in justify a full time beta.

lojack5 commented 8 years ago

I probably won't be back til the 2nd or so of January spending on weather (visiting relatives)

DianaNites commented 8 years ago

How can i best get back into actually doing something?

Utumno commented 8 years ago

@crazykilla15: and a happy new year :) I'd say set up a branch and factor out duplicate code in mod_links import*\ as I do in https://github.com/wrye-bash/wrye-bash/commit/a4e837f6262507828940e339472b9fb10bec1ed0

And stay tuned - the stuff I work on is really complicated at the moment but when I start merging the patchers in I will need some help squashing warnings.

I will merge the argparse stuff you did ASAP (but leave the FIXMEs out to be further investigated).

You should also rebase your branches over dev - if you don't manage let me know and I will help. You could then try and fix some exception pass you already have commented on - btw I fixed some so expect conflicts with your fixme comments - and I am investigating: http://stackoverflow.com/q/33650696/281545

Utumno commented 8 years ago

@iaz3: I saved our discussion for rebase etc here: 20160124160958.zip so I unclatter this issue. Thanks for opening an issue at your repo for that.

Status report: stuck on a bug reported by @Supierce and on a crappy laptop (that however is an opportunity to test some important code paths). Not bad ;)

Utumno commented 7 years ago

Well - getting there:

Utumno commented 7 years ago

@lojack5 @D4id4los

Release time. I have finally managed to solve some long standing BAIN issues but there are a couple things I want in for 307 and I need your help:

There are smaller stuff also @D4id4los - for instance:

Another very useful contribution would be editing the source code wiki now that you are familiarizing yourself with the code - it's badly out of date: https://github.com/wrye-bash/wrye-bash/wiki/[dev]-Deciphering-Source-Filenames

Finally @lojack5 @D4id4los - we need tests. We have a test repo set up, time to write some tests - any and every little tests helps - in particular BAIN and the patchers, but also the inis.

Utumno commented 7 years ago

@D4id4los - went ahead and deleted branches utumno-scandir and daidalos-229-mustbeactiveifimported-tag, changes are in utumno-wip and soon on dev - also @nycz nycz-exceptions-module (on dev)

Utumno commented 7 years ago

We should exclude scripts folder from python dist - confuses users: https://afkmods.com/index.php?/topic/4966-wrye-bash-all-games/&do=findComment&comment=168900

Utumno commented 6 years ago

Ok some improvements for packaging scripts:

Arthmoor commented 6 years ago

If the only point of the scripts folder is to be able to build the distribution packages then I see no problem with removing them from the packages intended for end user consumption.

Utumno commented 5 years ago

@GandaG @CptMcSplody @Infernio:

I would suggest that we post the nightly builds up on nexus - this way we reach much more people and we stop having people in the forums/discord that still come up with bugs in latest beta that are long since solved in the nightlies. What you say

I would also highjack this issue for some repo/branches cleanup as @CptMcSplody suggested - we probably do not need the meta and dev tools repos and most of the branches here.

I am waiting on your feedback

Arthmoor commented 5 years ago

If I may suggest, since Nexus upload isn't the fastest or easiest thing to work with on a regular basis, perhaps only uploading a "nightly" every 2 weeks or so would work better? That way those of us who are in a position to update more regularly can just do so as long as we know which branch to grab from off of GitHub and they'll have been battle tested a bit before going on Nexus.

Utumno commented 5 years ago

Totally agree @Arthmoor that's what I had in mind. Now the problem is that my and sharlikran's wips have diverged a bit - so I would need @Sharlikran and @CptMcSplody help to merge as much as possible and upload first nightly on nexus.

Or do you think we could upload the one that is on discord as is now to stop people from posting about solved issues?

Infernio commented 5 years ago

More frequent releases on the Nexus is definitely worth it - I'd say we simply put up the WIPs on the Nexus (ever 2 weeks or so, as Arthmoor said) the same way we currently do on the forums (AFKMods, etc.).

As for which one to upload - I'd wager that ESL support (i.e. marking as ESL-capable, ESL flipping in the mod list) is probably a pretty important feature for many people, so maybe the next uploaded build should at least have that?

GandaG commented 5 years ago

I'm almost done automating the upload process, just need to figure out the damn captcha - for some reason it forces me to spend wayyy longer (up to 2-3 mins) solving those things. captcha has been defeated.

Arthmoor commented 5 years ago

I think the branch with both Sharlirkan's ESL support and the FOMOD installer code should be the first one we offer up. That's a big thing we've been poking at that a lot of people will take notice of.

Utumno commented 5 years ago

Thanks for feedback @GandaG , @Infernio :)

@Arthmoor, yes I agree esls and fomods, I would however like to have the record updates in too. Re: fomod, where are we @GandaG ?

Automating the uploads ?! 👍 where is that?

Arthmoor commented 5 years ago

Which record updates? I don't recall seeing anything about that anywhere.

Utumno commented 5 years ago

Oh there is a bunch of record updates: https://github.com/Wrye-Code-Collection/wrye-bash/commits/wip-eslsupport-imports

GandaG commented 5 years ago

@Utumno Re: fomod - besides review, the big thing missing is pickling the fomod variables. I should also have added a final page on the fomod installer to ask the user whether to install (similar to the wizard) but I've been procrastinating on that (part because it's gui work, part because Im having so much fun with this upload stuff) :/

Re: automation - captcha has been defeated but you still have to go through with it the first time you login, after that I save the necessary cookies. There are a few things I need:

Utumno commented 5 years ago

whether to upload all or upload only the installer (or any combination)

all three - have a look at how files are now

file category (I'm guessing optional files)

doesn't it have "nightly"? :P

file description

have a look at current uploads

any other special requests

Yep - post those to the meta repo I would say that we should completely clean up - all the second posts stuff must go for instance

Utumno commented 5 years ago

Re: fomod pickling - you have to redefine your vars to be pickled in the extras_dict - search commit messages to see when this was introduced. Basically you can't add new persistent attributes to the installer without breaking existing Installer.dat and this is big nono. We will revisit all this for 308 (see that milestone issues) but for now this is the way to go

Utumno commented 5 years ago

And two other small points @GandaG - boop reminds of well you know so probably rename to fomods.py and also Installer is not a good idea - FomodInstaller. The sooner I have the final the sooner I can debug/review :P

GandaG commented 5 years ago

doesn't it have "nightly"? :P

I wish 😅

Yep - post those to the meta repo I would say that we should completely clean up - all the second posts stuff must go for instance

Those? The script? You lost me here, sorry :/

Utumno commented 5 years ago

I wish 😅

Updates section I'd guess

Those? The script? You lost me here, sorry :/

The script you wrote (for uploads) belongs to the meta repo I think (which needs a good deal of cleanup)

Infernio commented 5 years ago

So, I just had another encounter with someone who was stumped as to where the 'merging into the Bashed Patch' feature went.

If this is already happening this often with a Github-Fork / Discord-exclusive beta, then how bad will it be once we release on Nexus? At least for 307 itself we should find some way to tell users where that functionality went, preferably from within WB itself.

Utumno commented 5 years ago

Well - that's on Sharlikran's branch and I have asked on discord for a checklist with the changes that I have not seen yet so I have to fish them in the code and never be sure that all changes needed are addressed. So @Arthmoor , @CptMcSplody what should I include in my branch (esl coloring, is there, load order changes there, the Flip Esl button is there but we should enable it only on mods that is correct to do it, and I have no code for that). But what about the merging? What exactly should the code do?

Arthmoor commented 5 years ago

I can't give you a technical breakdown of what that code does and why it does it, all I can say for sure is that the ESL support has been extensively tested by many people, including several I've pointed toward that branch from places like Reddit.

IMO, trust the code. It works. I know you don't like to do that with code you haven't thoroughly examined but this is an important thing to get in place for SSE and FO4.

FelesNoctis commented 5 years ago

I'd echo Arthmoor on the necessity of getting the ESL support back in there ASAP. I keep a copy of Sharlikran's build installed specifically for the purpose of reliable and quick finding and flagging plugins. It's good for the less experienced modders who don't know what to look for. They don't need to know any extra steps, just check and ESLify. Without it a lot of people would be clueless how to begin. I've used the feature extensively and it's yet to steer me wrong.

MacSplody commented 5 years ago

But what about the merging? What exactly should the code do?

Merging has become redundant with ESL flagging. At least merging plug-ins into the bash patch.

what should I include in my branch (esl coloring, is there, load order changes there, the Flip Esl button is there but we should enable it only on mods that is correct to do it,

The changes made have effectively swapped merging for eslify. The functionality is very similar to how merging is checked, @Sharlikran Did a good job on making the changes and has made sure that any shortcomings in the record definitions are accounted for when converting files to ESL.

FelesNoctis commented 5 years ago

If clarify that to "replaced for most users". There are some extreme cases as we've discovered that there does appear to be a data cap (which may actually be lower than we originally thought), but the average modder would never hit that. Regardless, I'd say that's a good solution, swapping the merge for ESLify.

Utumno commented 5 years ago

Thanks, everybody - all I am looking for is a checklist as in:

The code has some rough edges and I need someone to add the info to the readme

Please @CptMcSplody @Arthmoor this is a priority so let's get this done by tomorrow (the checklist and the readme entry). The code has some rough edges @Arthmoor and I can't include it as it is but I managed to get most of it done, it's in my branch please use that. What is missing is the merging but I need that checklist. So what you expect to see in the UI - let the technical part to me (and Sharlikran)

Also re: ModFlipEsl menu, shouldn't we add a check to see that the mods we flip are indeed eslable

Also @Sharlikran dropped reinstated mergeable for fallout4 - with a note we need all records to be decoded. This code is simply not ready for dev - it's 90% though

Have a look everybody at the readme by @FelesNoctis - you should bnot edit the gitignore for files that should not be present in the repo, add those to your .git/info/exclude

There is another problem with this moving the files around - namely they should be added to the nsis installer cleanup - @fireundubh this is probably for you

Thanks @CptMcSplody for the new commits on mcs branch - are those ready for dev? Re: SVR support - not now definitely, much other to do, plus to do this we must finalize game handling refactoring. (@GandaG looking at you :))

MacSplody commented 5 years ago

@Utumno Yes those are ready, I was going to push it to dev-staging.

Utumno commented 5 years ago

While doing so please edit the commit messages slightly - some of them have leading whitespace or too big lines. Thanks!

Utumno commented 5 years ago

And I was wondering what this dev staging was for :P

Now once we push those could you do me a favour @CptMcSplody ? I pushed the branch by @Sharlikran in wip-eslsupport-imports

Could you redo that rebase so we verify I did not left anything out?

Utumno commented 5 years ago

Also @CptMcSplody is the adv-readme-updates branch good to go - if yes please delete it from here and let me know

MacSplody commented 5 years ago

is the adv-readme-updates branch good to go - if yes please delete it from here and let me know

Not sure, having dinner soonish, will have a look after it.

Could you redo that rebase so we verify I did not left anything out?

same ^^

@Utumno I updated those commit messages, so let me know if it needs anything else changed. Rebase or squash it onto dev-staging btw?

Utumno commented 5 years ago

Thanks - squash last two (docstring only commits are kind of noise) and push directly to dev I'd say :)

Then I will rebase over dev my branch as some of the wip esl content will be now in dev and let you double check. Goal of this wip-esl check is to have as much of the records work by @Sharlikran in dev asap

Bonne appetit!

Arthmoor commented 5 years ago

You need to keep in mind I don't know what any of this code does. Python to me is like trying to read a doctor's chicken scratch prescription. Occasionally something makes sense, but mostly it's just gibberish.

This to me is also being compounded right now by Bash having far too many branches and forks to know what code is being tested where and which branch we should be using to do this with.

If you're asking me what should be kept, you already know. The ESL support. Merging is an archaic construct in SSE/FO4 so there should be no more reason for having the "mark mergable" feature since it's obsolete and there should be no more "merge patches" feature for building the Bashed Patch. That feature has never been of as much use in Skyrim and above as it was for the older games anyway so it's not going to be missed if it's gone.

Utumno commented 5 years ago

@Arthmoor I am not asking you to go through the python code - all I am asking is a comprehensive list of what you should see changed in Bash as a user - so which menus, messages etc should be changed - so I can double check all is in the code

Utumno commented 5 years ago

Thanks for pushing @CptMcSplody - I pushed utumno-wip but could not rebase wip-esl due to conflicts in cell patcher I could not resolve - I leave this to you

Utumno commented 5 years ago

Also I noticed you @CptMcSplody or Sharl cleaned up the wrye-code-collection repo so I spent the day cleaning up my copies of those branches - I suspect that dev-patcher-updates and dev-modgroups must go too, no content in them that is not in dev-skyrim-updates that probably should go too as is absorbed into wip-esls - am I right?

Arthmoor commented 5 years ago

And I need to know for sure at this point what branch you want all this checked on because I no longer know. There's just too many of them to keep track of.

Utumno commented 5 years ago

Finally thanks @Arthmoor - I have been asking for this for two weeks, lost to coding My branch will notify you when I push these, but exactly because of all these branches I spent the whole day today cleaning up - if I had this list and clean branches I would have implemented this instead of having a headache and having written no code, duh

MacSplody commented 5 years ago

@Utumno I have the all the changes on the wip-esl branch ready just need to dissect it a bit.

image

Utumno commented 5 years ago

@Arthmoor esl changes pushed to utumno-wip here - please test :)

Arthmoor commented 5 years ago

I'm gonna check the boxes for what works.

"Check ESL Qualifications" on a file spits back this error:

Traceback (most recent call last):
  File "bash\balt.py", line 2505, in __Execute
    self.Execute()
  File "bash\balt.py", line 1605, in _conversation_wrapper
    return func(*args, **kwargs)
  File "bash\basher\mod_links.py", line 875, in Execute
    doCBash=self.doCBash, verbose=True)
  File "bash\bosh\__init__.py", line 2086, in rescanMergeable
    return self._rescanMergeable(names, prog, doCBash, verbose)
  File "bash\bosh\__init__.py", line 2111, in _rescanMergeable
    canMerge = is_mergeable(fileInfo, self, verbose)
  File "bash\bosh\_mergeability.py", line 159, in hasHighForms
    modFile)
  File "bash\bosh\_mergeability.py", line 142, in check_esl_topsSkipped
    reasons.append(u'\n.    ' + _(u'Record type: ') + u', '.join(sorted(
AttributeError: 'bool' object has no attribute 'append'

The "ESLify Self" option on files marked in green is greyed out sometimes, available in others, and I can'd discern a reason why.

The same is true for the option you marked as "Drop ESL Flag". Sometimes the option is available, other times it's greyed out, and there seems to be no pattern as to why.