Sometimes Compile fails to copy over Dependencies correctly

[Nuc1eoN]

I have found that many of the /Program/*/Resourses/Dependencies files on a default gobolinux install are erroneous. While inside /Data/Compile/Recipes/*/Resources/Dependecies they are in their correct form.

They might look like this (/Programs/Htop/3.0.4/Resources/Dependencies):

NcursesW >= 6.2
 %    

In some cases Dependencies are actually missing, but in every case (the bug happens) there is an % at the end of the file. Or at least that is, how cat displays it... in nano it's just blank. Also it seems Runner has issues parsing those files.

A reproducible Recipe for the bug seems to be Htop 3.0.4. Everytime I do Compile htop 3.0.4, its Dependencies file looks malformed as above. I'm on Compile git btw.

But many already preinstalled packages seem to be affected to. Confirmed this with @ermo

[Nuc1eoN]

@lucasvr Would you be able to take a look? I find this one pretty nasty of a bug, especially if we want to give runner/--pure some testing :D

[sage-etcher]

In some cases Dependencies are actually missing, but in every case (the bug happens) there is an % at the end of the file. Or at least that is, how cat displays it... in nano it's just blank.

The converted Dependency file ends with a space 0x20, rather than the more standard linefeed 0x0A. When the file is printed to the terminal, the last character sent isn't a linefeed, so the shell displays a % to indicate such; the same can be seen with printf "hello world". Luckily, the % isn't piped along with the output, so it shouldn't cause problems in most scripts.

I traced the cause back to the CheckDependencies script, line 807:

Compile: bin/Compile: compile_recipe(): 875:
   install_extras || wrap_fail "Failed installing files."

Compile: Functions/Compile: install_extras(): 506:
   CheckDependencies --mode=convert --file $recipedir/Resources/Dependencies | tee "$installprefix/Resources/Dependencies" >/dev/null

Scripts: bin/CheckDependencies: 807:
   print(ConvertRecipeToBinary(p, v, r, t, u, depfile=depfile), end=' ')

Throughout Gobo's scripts, it seems many python print calls specify end=' ', what is the logic behind this? is it important?

For this issue, one potential fix is to remove , end=' ' from the print function in question, nipping it at the source. However, if the removal would cause issues, we could also handle it in install_extras() by piping the output from CheckDependencies into sed 's/^ *$// before sending it off to tee.

I'm unsure what use the end=' ' serves, some guidance would be greatly appreciated.

A reproducible Recipe for the bug seems to be Htop 3.0.4. Everytime I do Compile htop 3.0.4, its Dependencies file looks malformed as above. I'm on Compile git btw.

Also, I'm unsure what part you are referring to as malformed? The /Programs/*/*/Resources/Dependencies file is generated by CheckDependencies --mode=convert.

It's my understanding that the --mode=convert flag is used to re-write file using the dependency versions that are currently installed. The loop skips any dependencies that it cannot find on the live system.

The full implementation is in the ConvertRecipeToBinary() function, of the CheckDependencies script. (Scripts repo)

[Nuc1eoN]

Good job on tracking it down!

Luckily, the % isn't piped along with the output, so it shouldn't cause problems in most scripts.

Interesting, so apparently it is not as bad as I have suspected. (However I did have some issues with Runner not importing some dependencies correctly, and I suspected it is connected to this bug.)

Throughout Gobo's scripts, it seems many python print calls specify end=' ', what is the logic behind this? is it important? [...] I'm unsure what use the end=' ' serves, some guidance would be greatly appreciated.

Good question. Maybe @lucasvr or @mwh has a clue.

[lucasvr]

That's @detsch legacy :)

[detsch]

Looking at commit 6d2f24973cb40568d98da42137b80184be711872, which converted the script to Python 3:

-                       print ConvertRecipeToBinary(p, v, r, t, u, depfile=depfile),
+                       print(ConvertRecipeToBinary(p, v, r, t, u, depfile=depfile), end=' ')

Most likely the , was added in the original code (Python 2) just to prevent a new line to be printed. end='' should work fine.