jbang wrapper generated files have wrong timestamp

[manikmagar]

I upgraded jbang to 0.45.0 and tried jbang wrapper install. It generated jbang command files but the file timestamp was 80's.

Also, the file browser is showing same -

Java version -

openjdk version "14.0.2" 2020-07-14
OpenJDK Runtime Environment AdoptOpenJDK (build 14.0.2+12)
OpenJDK 64-Bit Server VM AdoptOpenJDK (build 14.0.2+12, mixed mode, sharing)

I tried in two different folders and had same behavior for generated files.

Let me know if any other information is needed.

[maxandersen]

/cc @quintesse

[manikmagar]

Ran with JDK 11 just in case but had same behavior -

[chrsblck]

@maxandersen Can I work on this?

It's probably something around the StandardCopyOption.COPY_ATTRIBUTES https://github.com/jbangdev/jbang/blob/6657150f510bb398746fb6d5e7820a5c88d55d61/src/main/java/dev/jbang/cli/Wrapper.java#L71-L74

[maxandersen]

Go ahead! Work away :)

[chrsblck]

@maxandersen Do you have any suggestions here?

For example, the Files#copy docs for COPY_ATTRIBUTES state:

Attempts to copy the file attributes associated with this file to the target file. The exact file attributes that are copied is platform and file system dependent and therefore unspecified. Minimally, the last-modified-time is copied to the target file if supported by both the source and target file store. Copying of file timestamps may result in precision loss.

Seems like we can just remove this option, possibly. The other fix would probably be making a call to update the modified-timestamp after copy. wdyt?

[maxandersen]

My understand is that if you use StandardCopyOption.COPY_ATTRIBUTES best effort is done and last modified at least should be right.

Haven't tested it though.

[chrsblck]

Actually, I couldn't replicate locally, so I installed via Homebrew.

drwxr-xr-x    - cblack  9 Oct 12:47 /usr/local/opt/jbang/libexec
drwxr-xr-x    - cblack  9 Oct 12:47 ├── bin
.rwxr-xr-x 5.4k cblack  9 Oct 12:47 │  ├── jbang
.rw-r--r-- 5.0k cblack  1 Feb  1980 │  ├── jbang.cmd
.rw-r--r-- 5.9M cblack  1 Feb  1980 │  ├── jbang.jar
.rw-r--r-- 6.6k cblack  1 Feb  1980 │  └── jbang.ps1
drwxr-xr-x    - cblack  1 Feb  1980 └── examples
.rw-r--r-- 6.2k cblack  1 Feb  1980    ├── analytics.java
...

It looks like this is the issue.

Update I'm going to look for where the Homebrew stuff is bundled. Maybe the build or Homebrew repo (idk how Homebrew works actually 😬 ) isn't syncing time correctly

[chrsblck]

Just for fun I checked older releases, they seem to have this issue too

Permissions Size User   Date Modified Name
drwxr-xr-x     - cblack  1 Feb  1980  bin
drwxr-xr-x     - cblack  1 Feb  1980  examples

[maxandersen]

Oh so the brew install doesn't do proper time modifications? Weird.

Search for .rb or look in home tap Repo.

[chrsblck]

Sorry, I wasn't clear. I checked the older releases by downloading the zip files from Github.

For example

❯ export VERSION='0.34.0' && curl -sLO https://github.com/jbangdev/jbang/releases/download/v$VERSION/jbang-$VERSION.zip

~/Desktop/jbang
❯ unzip -q jbang-0.34.0.zip

~/Desktop/jbang
❯ la -lh jbang-0.34.0
Permissions Size User   Date Modified Name
drwxr-xr-x     - cblack  1 Feb  1980  bin
drwxr-xr-x     - cblack  1 Feb  1980  examples

So I guess it's Github Actions, that seems unlikely though 🤔 . Still looking...

[maxandersen]

ah - so I can see it is gradle that does this . all zips have the same exact date. Believe that is done to ensure the zips stays the same between builds.

I'm wondering if we could at least set timestamp to the last commit it is built from. That would be more correct and still stable.

[maxandersen]

/cc @aalmiray do you know of a trick here or does all software from gradle look like something from the 80'ies ? :)

[aalmiray]

Could it be related to https://docs.gradle.org/current/dsl/org.gradle.api.tasks.bundling.Zip.html#org.gradle.api.tasks.bundling.Zip:preserveFileTimestamps ?

[maxandersen]

yeah, so I actually set that in build.gradle .... to have the zip stay the same no matter how often I rebuild...wonder if can do one better...and configure the timestamp used rather than 1980ies..

[maxandersen]

I guess we would need to set the timestamp somehow on all incoming files...somehow.

[chrsblck]

Sounds good, I'll look into different options and report back (after lunch 😄 )

[chrsblck]

@maxandersen I found this https://discuss.gradle.org/t/better-alternative-for-preservefiletimestamps-false/34528.

I'm not very familiar with Gradle, we use Maven, do you have easy steps to reproduce the build issue you're solving by setting preserveFileTimestamps = false?

[maxandersen]

what happens is that a rebuild will create a different jar even though nothing changed. that makes things slower and mutable. I'll rather keep the 1980 "feature" than breaking reproducible build if the only issue is this "surprise".

i'm no expert in gradle neither so would need some exploration to see how can set the timestamps properly.

[aalmiray]

Yeah, I don't think you can get around this if you want to enforce reproducible builds.

[maxandersen]

if we set the files to last commit timestamp should give the same result should it not?

[chrsblck]

I'm looking, but I don't see support for setting this timestamp.

[aalmiray]

I suppose so, as long as you can fix the timestamp to a particular value that makes sense to you.

[chrsblck]

I'm not seeing a way to do this on the gradle side. Maybe gradle has a low level api or something?

[chrsblck]

@maxandersen Sorry, I haven't had time to look into this again, but I should tonight I hope. Do you have any suggestions on how to proceed?

[maxandersen]

the thing to realize is that build.gradle is just a groovy DSL and you have access to almost all groovy features. Thus one way I can think of it it is simply add a task or afterLoad section that iterates the relevant files collection and do a call to Files.setAttribute(p, "creationTime", FileTime.fromMillis(c.getTimeInMillis()));.

The tricky part is to find the right timestamp to use - here I would go search for a gradle plugin that should be able to have already handled getting git info including timestamp for latest commit.

That would be the bruteforce way to start exploring.

[aalmiray]

https://github.com/nemerosa/versioning/ is my preferred choice for git in Gradle, sadly for for this case it may not work given that it does no give you access to the commit date as far as I can tell.

https://github.com/ajoberstar/gradle-git-publish is the other one I rely on a lot. It uses https://github.com/ajoberstar/grgit which wraps jgit, thus you may be able to get the date form it. Or simply use a Java process to invoke git underneath ;-)

[chrsblck]

Hi @maxandersen - Sorry for the delay, I've not had a large enough block of time to get started on this yet. I end up down a rabbit hole trying to figure out how Gradle works (I've never used this tool) and where to start (and give up or run out of time). Would you be able to give me a hint where to start?

Update: For example, I've found minimal docs for grgit, but I have no clue where to wire these in. Or I know I can run gradlew spotlessApply, but I see no definition for spotlessApply :/

Sorry, these are probably stupid questions haha

[maxandersen]

Plugins can contribute these tasks.

Spotless plugin (see near top) makes them available.

So for grgit add it as plugin.

Then look at "tag" task. That's more.or less just groovy.code that can do whatever it needs to do.

In this case iterate over all files in the build install dir. There is a property for it but I'm only on mobile atm :)

Hope that helps.