search

inpta / pinta

Data analysis pipeline for upgraded GMRT pulsar data
GNU General Public License v2.0
3 stars 0 forks source link

Regarding log files #11

Closed SaiC1999 closed 3 years ago

SaiC1999 commented 4 years ago

I have tried using --log-to-file option. I couldn't find out what the error was when the pipeline crashed. But after spending some time I figured out that there was not enough disk space in the working directory. It would be better if we can find out the error by seeing the log file.

Seeing this I printed the output in one of the files and found out the error there but it was showing something way different than what I expected. It was "File not found error" because it couldn't remove the file which was not created due to lack of space. It would be great if we can print out a message saying there was not enough disk space so that user can figure out the problem easily.

Screenshot 2020-07-20 at 9 57 35 PM
ymaan4 commented 4 years ago

A better way would be if the pipeline itself checks for the remaining space in the working directory and exits with a message if the space is less than N times the input file size. It should be pretty easy to implement, and N=5 will suffice I believe.

On Mon, Jul 20, 2020 at 6:29 PM SaiC1999 notifications@github.com wrote:

I have tried using --log-to-file option. I couldn't find out what the error was when the pipeline crashed. But after spending some time I figured out that there was not enough disk space in the working directory. It would be better if we can find out the error by seeing the log file.

Seeing this I printed the output in one of the files and found out the error there but it was showing something way different than what I expected. It was "File not found error" because it couldn't remove the file which was not created due to lack of space. It would be great if we can print out a message saying there was not enough disk space so that user can figure out the problem easily. [image: Screenshot 2020-07-20 at 9 57 35 PM] https://user-images.githubusercontent.com/68557345/87962004-2de3c200-cad4-11ea-8f13-75dde5175bac.png

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/abhisrkckl/pinta/issues/11, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABXWLABAYFDK3LOWT4FKKP3R4RWG3ANCNFSM4PCQTKXQ .

SaiC1999 commented 4 years ago

Yeah, That would also be great. And log files created by using the inbuilt option in the pipeline do not show the error. Any reason why?

abhisrkckl commented 4 years ago

I think what we need here is not "disk space remaining", but "disk space remaining accessible to the current user". I don't know of any way to do the latter. I will try to find out.

tprabu2000 commented 4 years ago

I think that quota -s can reliably report the disk space available for the user. But not sure if there are constraints using in the pipeline!

abhisrkckl commented 3 years ago

We have decided not to implement this in the pipeline. It is better if the users manually check their quota before running the pipeline. This point will be added to the user guide.