search

DocNow / twarc

A command line tool (and Python library) for archiving Twitter JSON
https://twarc-project.readthedocs.io
MIT License
1.37k stars 255 forks source link

Lockfile message with utils\twarc-archive.py command #308

Open jason-lange opened 4 years ago

jason-lange commented 4 years ago

Hi, I'm trying to do a case insensitive keyword search for "census" and then archive the search, so as to build a running archive of tweets over the coming weeks. I created a folder called census and in Windows CMD I executed this command in my wd: utils\twarc-archive.py census \census I keep getting this message: Another twarc-archive.py process with pid 26068 is running. If the process is no longer active then it may have been interrupted. In that case remove the 'lockfile' in \census and run the command again. I don't see any lockfile, don't see that pid running in task manager's services tab, and got the command even after rebooting my machine (!). I saw some folks have raised the matter of lockfiles around twarc-archive but I don't see anyone having this particular problem. Any guidance much appreciated!

edsu commented 4 years ago

I'm curious, why you are using \census instead of utils\twarc-archive.py census census. Do you see a directory called \census ?

jason-lange commented 4 years ago

Thanks @edsu . I can get into the census directory by running C:\Users\XXFOLDER_NAMEXX\OneDrive - Thomson Reuters Incorporated\Documents\twarc-master\census I'm new to command line tools and was following the the example in the twarc-archive file, which puts a \ at the beginning of the file path: EXAMPLE: So for example if you want to search for tweets mentioning "ferguson" you can run it: % twarc-archive.py ferguson /mnt/tweets/ferguson When I run utils\twarc-archive.py census census I get a message "no such profile main in C:\Users...."

edsu commented 4 years ago

Oh ok. So you are actually running something like:

utils\twarc-archive.py census C:\Users\XXFOLDER_NAMEXX\OneDrive - Thomson Reuters Incorporated\Documents\twarc-master\census

?

jason-lange commented 4 years ago

I suppose so. At risk of providing too much detail, I start out changing directories to a twarc-master folder, running in the CMD window: cd C:\Users\u8013352\OneDrive - Thomson Reuters Incorporated\Documents\twarc-master> Once in there, I am able to run a search command like the README example: twarc search blacklivesmatter > search.jsonl And it produces a json file in twarc-master as expected. I've then tried creating folders called census in both twarc-master and in twarc-master's utils folder, before running this command: utils\twarc-archive.py census \census And it tells me: Another twarc-archive.py process with pid 26068 is running. If the process is no longer active then it may have been interrupted. In that case remove the 'lockfile' in \census and run the command again. If I try putting any a folder as the destination that I'm sure exists, like one further up the file tree, and running the same command I get: C:\Users\u8013352\OneDrive - Thomson Reuters Incorporated\Documents\twarc-master>utils\twarc-archive.py census \Users\u8013352 Another twarc-archive.py process with pid 22160 is running. If the process is no longer active then it may have been interrupted. In that case remove the 'lockfile' in \Users\u8013352 and run the command again. Interestingly, a lockfile document and archfile.log appears in \Users\u8013352 at that point.