FieldData being output with non-standard file names and unknown file type

[danrhouck]

I don't know what I could have done, but now my simulations are outputting the field data with file names like V27NOS~1, V27NOS~2, etc. Those are the first six characters of my job title, so it's not random, but not the typical <input file name>_FieldData_<timestep>.csv. After it got to V27NOS~5, it switched to TI31000~, TI31001~,...TI31004~, then KBBD0KP~ and other seemingly random names. I have no explanation for the latter two. They still work in that I can open them and process them in Matlab, so I guess it's not a problem, but it's odd if not concerning.

[danrhouck]

Just an update: This only happened for one simulation, however, it repeated when I ran it again. Others don't seem affected by whatever this is.

[danrhouck]

Something like this happened again. This time, however, instead of all the FieldData files I expected to get, I got only one file that had no extension, was not named with the usual format, and was much too small to contain all the data. The filename appeared to be cut short, and the input filename was longer than usual. I wonder if there's a bug caused by exceeding the expected filename length? I don't think this accounts for what happened above, but perhaps these issues are related.

[whophil]

Please provide more information such as: input files, the full path of your working directory, the operating system, what version (commit hash) of CACTUS you have compiled, etc.

[danrhouck]

I've attached an input file (saved as .txt so I could upload here) that produced mostly randomly named Field Data files without extensions. I'm running the most recent version of CACTUS available from a CEE projects folder on the network. The simulations are being run on a Linux compute server. Let me know if you need more specific info.

I have another sim with an input file that is one character shorter than this one. It's Field Data saved with the correct file names, but the extension is shortened to just .cs. I also tried one that is one character longer than the attached. With it, I only got four out of the 25 Field Data files that should have been saved and their file names are truncated by one digit and have no extension. Not exactly a pattern with these, but I suspect it's related to input file name length. I'm not sure which sim caused me to post the first time, but the one I think it was also has a "long" file name. V27nr200nti25y5.txt

[danrhouck]

I'm pretty convinced now that there is a 23 character limit to the Field Data files. The "FieldData" part uses up 11 of those characters. Depending on how long your simulation is and when you start saving, that can vary the appended file number (e.g., 010, 0100, 01000, etc.). Seems like the code will always use your input filename, so, if there aren't enough characters left to append the full timestep number, it will stop saving and say nothing. Your simulation will end, but you will not have saved all the data you wanted. So final thing to know is this: 12 - number of characters in the input filename = longest timestep number to be saved plus one for the leading zero.

[danrhouck]

I thought I had this sorted out, but apparently not. Even with input file names that are the same length and (I thought) short enough to avoid the truncation issues detailed above, I'm still getting inconsistent file naming of the Field Data. Sometimes the file extension is being cut off, sometimes the last digit of the file number/timestep (which can then limit how much is saved), and still sometimes they are named with seemingly random combinations of characters.

Update to the weirdness: Somehow Matlab can "see" the more or less correct (no file extension, but otherwise right) file names that show up as random combos of characters in the folder view.

I would really appreciate some help with this!

[whophil]

Are you viewing these files over a network share? If so, do the files have bad names when viewed on the Linux machine, or just when you view them over the network share?

[danrhouck]

You’re right. All the names are correct when viewed through the Linux machine and only some are wrong when viewed on my machine looking at the network folder. That’s weird. Can you explain that at all? I guess that explains why Matlab can still correctly read in the files too.

[whophil]

Windows does wacky things when it encounters files whose absolute paths are over 255 characters in length. Things are extra complicated because the files you are viewing over the network are likely shared over the Samba protocol, which may or may not also have filename length issues.

I have read that Windows applications can support longer filenames, but haven't seen this in the wild. Perhaps that is why Matlab is able to read them just fine.

I'm going to close this as it seems this is not a CACTUS issue.

[danrhouck]

Just a final word of caution on this topic: This is especially an issue when running long (many timesteps) simulations and saving data at the end because the timestep length adds more characters to the filenames. If their aren't enough characters left, it appears that new files get saved over existing ones. For example, timestep 5000 might get saved as simulation_FieldData_0500 if it runs out of characters. Then, all timesteps from 5000 through 5009 will have the same name and not be saved as distinct files.

[whophil]

Just a final word of caution on this topic: This is especially an issue when running long (many timesteps) simulations and saving data at the end because the timestep length adds more characters to the filenames. If their aren't enough characters left, it appears that new files get saved over existing ones. For example, timestep 5000 might get saved as simulation_FieldData_0500 if it runs out of characters. Then, all timesteps from 5000 through 5009 will have the same name and not be saved as distinct files.

On Windows