search

Reading-eScience-Centre / ncwms

ncWMS - A Web Map Service for displaying environmental data over the web
Other
63 stars 30 forks source link

how to prevent nc files are locked by ncWMS? #31

Open matthijslemans opened 6 years ago

matthijslemans commented 6 years ago

I use a ncWMS hosted in Tomcat. The netcdf files are created by another application. This application exports the netcdf files to a file directory every hour, with same file name so files are overwritten. This directory is also the place where ncWMS reads the netcdf files. This causes conflicts, because files cannot be deleted if the ncWMS service is running. What would be a good solution to work around this?

matthijslemans commented 6 years ago

The locked files are not visible, but the inode is still there. File handles are removed by killing the process that opened them, tomcat in this case. As a feasible workaround for now, I restart Tomcat every night with a cronjob, to let tomcat unlock the old files and release the used memory again.

guygriffiths commented 6 years ago

Sorry for the slow response on this, I've been busy with other projects. I'm taking a look into this now, but I'm not able to reproduce it.

I can successfully copy new data in place of an existing one (which is present in the NetcdfDataset cache, which keeps file handles open) and once it is refreshed - either automatically on a time interval or by manually refreshing - the new data is visible.

Can you give me any more information on your system and setup? e.g. ncWMS version, Tomcat version, OS, etc.

tewinkel commented 6 years ago

I also have this issue with NCWMS on windows 10 with tomcat 8.5, I tried versions NCWMS 2.2.13 and 2.4.12.

Even if the file is removed from NCWMS admin page the file is locked by tomcat and cannot be moved or deleted. The only solution I found is to restart tomcat to remove the file lock.

I think the problem is only on Windows because i never had problems on Ubuntu. Windows is probably a bit more stricter in checking if files are still in use before you can delete them.

matthijslemans commented 6 years ago

I enounter the issue on a Linux system, with tomcat version 7, and NCWMS version 2.4. I use a script that restarts Tomcat every night, in order to remove the file lock.

Thanks, Matthijs

2018-07-27 12:18 GMT+02:00 tewinkel notifications@github.com:

I also have this issue with NCWMS on windows 10 with tomcat 8.5, I tried versions NCWMS 2.2.13 and 2.4.12.

Even if the file is removed from NCWMS admin page the file is locked by tomcat and cannot be moved or deleted. The only solution I found is to restart tomcat to remove the file lock.

I think the problem is only on Windows because i never had problems on Ubuntu. Windows is probably a bit more stricter in checking if files are still in use before you can delete them.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Reading-eScience-Centre/ncwms/issues/31#issuecomment-408376994, or mute the thread https://github.com/notifications/unsubscribe-auth/AlyRSHLLv09T0I3zWCJIHdpYrHWxDnEgks5uKujrgaJpZM4UU5kH .

guygriffiths commented 6 years ago

@tewinkel - I've not been able to reproduce it on Linux, and Windows certainly has more restrictive file locking, so that was my assumption. But if @matthijslemans is seeing it on Linux, it seems that something else is going on. I'll look into it more when I have some free time.

acrosby commented 6 years ago

I would just like to chime in that we see this as well where ncwms is running in a container on a linux system, but the data is on a windows server accessed via a cifs/samba mount. It is only very occasionally that we run into the problem--usually from a process that attempts to clean up older files that are part of an aggregation. Restarting ncwms does not help in our case and we have to restart the windows server that hosts the data to get the inode back into sync. My guess is that a random thread has stalled out on an old request and is still trying to hold on to the file, but I don't know enough about java to optimize our chances to avoid the problem. Perhaps routine restarts are a sufficient work around to avoid the problem in the first place, so I will try to implement that for now.

Suheyb commented 4 years ago

I have same problem. Is there any solution except from restarting Tomcat?