script-output symlink stopped working on linux

[fireduck64]

Using: L0laapk3_FactorioMaps_3.5.5 System: Linux (Debian) Factorio: 0.18.3

Running: python3.6 ./auto.py --dayonly

Results: No save name passed. Using most recent save: Snail factorio path: /home/joseph/app/factorio/bin/x64/factorio output folder: script-output/FactorioMaps/Snail checking for updates enabling FactorioMaps mod cleaning up
building autorun.lua
building config.ini
starting factorio
Surface prescan Snail/4/nauvis
Surface capture Snail/4/nauvis/day (and here it hangs forever)

Running: python3.6 ./auto.py --dayonly --verbose --verbosegame

Results: ...snip.... [GAME] 16.847 Warning LuaGameScript.cpp:1779: take_screenshot error: filesystem error: create_directories "/tmp/FactorioMaps-986385054/script-output" failed: Not a directory [/tmp/FactorioMaps-986385054/script-output]
[GAME] 16.847 Warning LuaGameScript.cpp:1779: take_screenshot error: filesystem error: create_directories "/tmp/FactorioMaps-986385054/script-output" failed: Not a directory [/tmp/FactorioMaps-986385054/script-output]
[GAME] 16.847 Warning LuaGameScript.cpp:1779: take_screenshot error: filesystem error: create_directories "/tmp/FactorioMaps-986385054/script-output" failed: Not a directory [/tmp/FactorioMaps-986385054/script-output]
[GAME] 16.848 Warning LuaGameScript.cpp:2035: write_file error: filesystem error: create_directories "/tmp/FactorioMaps-986385054/script-output" failed: Not a directory [/tmp/FactorioMaps-986385054/script-output]
[GAME] 16.848 Warning LuaGameScript.cpp:2035: write_file error: filesystem error: create_directories "/tmp/FactorioMaps-986385054/script-output" failed: Not a directory [/tmp/FactorioMaps-986385054/script-output]
[GAME] 16.849 Warning LuaGameScript.cpp:2035: write_file error: filesystem error: create_directories "/tmp/FactorioMaps-986385054/script-output" failed: Not a directory [/tmp/FactorioMaps-986385054/script-output]
[GAME] 16.864 Warning LuaGameScript.cpp:2035: write_file error: filesystem error: create_directories "/tmp/FactorioMaps-986385054/script-output" failed: Not a directory [/tmp/FactorioMaps-986385054/script-output]

Inspecting that location: /tmp/FactorioMaps-986385054$ ls -la total 1512 drwxr-xr-x 3 joseph joseph 4096 Jan 31 10:15 . drwxrwxrwt 24 root root 16384 Jan 31 10:15 .. drwxr-xr-x 2 joseph joseph 4096 Jan 31 10:15 config -rw-r--r-- 1 joseph joseph 1458581 Jan 31 10:15 crop-cache.dat -rw-r--r-- 1 joseph joseph 54181 Jan 31 10:15 factorio-current.log -rw-r----- 1 joseph joseph 0 Jan 31 10:15 .lock -rw-r--r-- 1 joseph joseph 1759 Jan 31 10:15 player-data.json lrwxrwxrwx 1 joseph joseph 39 Jan 31 10:15 script-output -> /home/joseph/app/factorio/script-output

It appears something doesn't like treating a symlink as a directory?

[L0laapk3]

Hi, I'll look at the issue after next week when I have more time. In the meantime, can you try this soon-to-be merged PR? Some case sensitive stuff has been adressed for Linux specifically.

[fireduck64]

As a work-around, removing the symlink and making the directory real quick seems to be working.

Yeah, I can do whatever in about an half an hour. About to head into a meeting.

[L0laapk3]

Hi, did you get a chance to try the code from the PR?

[fireduck64]

I can't really test it, getting same error: [GAME] 19.659 Warning LuaGameScript.cpp:2038: write_file error: filesystem error: create_directories "/tmp/FactorioMaps-v6pq_mmb/script-output" failed: Not a directory [/tmp/FactorioMaps-v6pq_mmb/script-output]
[GAME] 19.685 Warning LuaGameScript.cpp:2038: write_file error: filesystem error: create_directories "/tmp/FactorioMaps-v6pq_mmb/script-output" failed: Not a directory [/tmp/FactorioMaps-v6pq_mmb/script-output]

And this is with 0.18.4 now.

[IIPoliII]

Can you please show us the permissions on the /tmp folder ll or ls -al and was the folder /tmp/FactorioMaps-v6pq_mmb created ? what is the permissions again of this folder?

Did you try using a sudo user ? Also for permissions are you the owner of the folders ?

Please give us the output for this it's maybe bounded to this.

Secondly are you using a symlink because you want to put it on another disk or another partition with a different filesystem?

PS : Did you tried to do some fat chmod -R 777 around the files/folders?

[fireduck64]

I'll send permissions in a bit. I am not using a symlink. I didn't create it, so either factorio is or the mod is.

On Thu, Feb 13, 2020, 04:13 Poli notifications@github.com wrote:

Can you please show us the permissions on the /tmp folder ll or ls -al and was the folder /tmp/FactorioMaps-v6pq_mmb created ? what is the permissions again of this folder?

Did you try using a sudo user ? Also for permissions are you the owner of the folders ?

Please give us the output for this it's maybe bounded to this.

Secondly are you using a symlink because you want to put it on another disk or another partition with a different filesystem?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/L0laapk3/FactorioMaps/issues/57?email_source=notifications&email_token=AAG5NSBRPCJ66JGBV6SEVH3RCU2O7A5CNFSM4KOLMPQ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELUXWEY#issuecomment-585726739, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAG5NSBA52OX2ZE4DQCH5CLRCU2O7ANCNFSM4KOLMPQQ .

[fireduck64]

Note: in auto.py taking out this line seems to make it work: linkDir(os.path.join(tmpDir, "script-output"), "../../script-output")

And by work, I mean it writes everything in a tmp location rather than script-output

It appears that some of the lua framework doesn't like that symlink.

[L0laapk3]

I've been looking into avoiding symlinks completely, see #59 for this. This is waiting on feature request on factorio's side, feel free to go voice your support there :)

However there is no guarantee that this will be implemented. However, I would still expect the current implementation to work just fine for you, seeing as it worked for other people on linux, so it looks like there is more going on.

[fireduck64]

My suspicion is that the Lua behavior got more strict in 0.18 but I'm not sure.

On Thu, Feb 13, 2020, 15:43 L0laapk3 notifications@github.com wrote:

I've been looking into avoiding symlinks completely, see #59 https://github.com/L0laapk3/FactorioMaps/issues/59 for this. This is waiting on feature request on factorio's side, feel free to go voice your support there :)

However there is no guarantee that this will be implemented. However, I would still expect the current implementation to work just fine for you, seeing as it worked for other people on linux, so it looks like there is more going on.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/L0laapk3/FactorioMaps/issues/57?email_source=notifications&email_token=AAG5NSA6YJ57V5WZC3TQG5LRCXLKHA5CNFSM4KOLMPQ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELXA6RA#issuecomment-586026820, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAG5NSBZ7UEMEAPMZIZX2ODRCXLKHANCNFSM4KOLMPQQ .

[IIPoliII]

^ Yes i think L0laapk3 even wrote a thing about it but it became a little bit more strict

[SuperTux88]

seeing as it worked for other people on linux, so it looks like there is more going on.

Are there other people where it worked with 0.18 on linux? What did they do different? I have the exact same problem.

Is there already a workaround for this? Just replacing the symlink with a directory probably doesn't work, because then the screenshots end up in the tmp directory but the rest of the factoriomaps script expects it to be in the real script-output folder where it then doesn't find it? Also the tmp directory is deleted at the end.

[SuperTux88]

OK, I hacked something, it's not pretty but worked. I used the TMPDIR env var to point to somewhere in the factorio folder and not to /tmp (because turns out my 32GB ramdisk is not nearly enough for all the screenshots). Then I create a script-output-folder in the temp folder and link this folder to the non-temp-script-output where the rest of the script expects all the files (so basically symlink the other way around, that way factorio has a real folder, and the rest of the script has a symlink, but the script doesn't care if it's a symlink or a folder), and I removed the cleanup of the temp from the script (because the temp folder is now the target, so I want to keep it).

That works, but is a really ugly hack (and it probably only works the first time, when the non-temp-script-output is still empty). But maybe a proper solution can be created in a similar way (so the script works in the temp folder, so no symlink is needed, but move everything to the real target folder after it's finished, so the temp folder can be deleted again)?

[maxsupermanhd]

Will this be fixed in some time?

[maxsupermanhd]

@SuperTux88 Can you please make a step by step instruction what you have made? I can not reverse this symlink, throws already exists error. Is this possible to fix this real quick to just not making any links, just move to target when done (by a script, not by hands)?

[L0laapk3]

I don't see a good, non-hacky solution to this.

I made a factorio feature request that will allow me to get rid of symlinks for good. So far it did not get any response, If this is something that interests you, please head over here, and let the devs know: https://forums.factorio.com/viewtopic.php?f=28&t=81221

[maxsupermanhd]

I don't see a good, non-hacky solution to this.

So what about bad and hacky solutions? If solution gonna work, i'll accept all bad things under the hood till Factorio team make this feature.

[maxsupermanhd]

Steps that made thing work:

1. set TMPDIR somewhere outside /tmp/ to the real directory. (idk there is no clear reason why it fails here for now)
2. Comment tempdir cleanup in auto.py
3. Run tool as expected with tmpdir envvar.
4. In factorio savefile enter editor to pause ticks (to physicly have chance to do step 5)
5. Wait till factorio bootstraps, enters savefile with editor and replace script-output link with ordinary folder.
6. Leave editor in factorio and let it generate screenshots. (Do all stuff till powup appears) (Do not close it manually after)
7. Manually move contents of script-output as auto.py excepts into original script-output dir (as link should do)
8. Auto.py will detect all files in place and continue execution as normally will do.

Is there way to implement moving files in script to avoid all hand work? Any 5-minute-crunch will do the job but i'm not as good in python as you guys.

[maxsupermanhd]

So no kinda crunch to reverse this? Idea: Expect everything in tmp dirs and then move everything away from that folder to proper location by mv. Can it be done with bashscript?

[L0laapk3]

To my understanding that won't work as the tmp mount isn't large enough to contain all the files. I strongly recommend to go voice your support on the feature request to set the desired location for script-output, so this symlink hack will finally stop being a problem: https://forums.factorio.com/viewtopic.php?f=28&t=81221

[skorokithakis]

@L0laapk3 can you come up with a reasonable workaround until Factorio implements the feature? Currently the script is unusable for us and it could be years until Factorio fixes this.

[fireduck64]

My workaround is to do the map export on windows. I use a mix of windows and linux desktops so not a big deal for me but I can see how that wouldn't work well for some.

[L0laapk3]

I'm not really too interested in writing temporary code only to have to throw it away soon after. I expected it (and still do) to be an easy feature to implement for factorio.

If someone wants to make a pull request for this, that would be an option, otherwise I recommend to voice your support on the above linked feature request thread.

[skorokithakis]

My workaround is to do the map export on windows

Yeah, unfortunately that doesn't work for me :(

[L0laapk3]

Theres light at the end of the tunnel guys, the feature I was waiting on should be implemented as of factorio 0.18.29 .