search

Sn0wCrack / ExHen-Archive

System for crawling/archiving and viewing ExHen galleries
The Unlicense
89 stars 8 forks source link

Audit task doesn't seem to work as intended #5

Closed Sunako closed 7 years ago

Sunako commented 7 years ago

From https://github.com/Sn0wCrack/ExHentai-Archive/blob/master/setup/Linux-Setup.md#cron

Audit - updates meta data for added galleries. Will typically update each gallery periodically for new tags, or add newer versions of that gallery to the database.

It does not seem to do any metadata or tag updating as it is apparently supposed to, it just rotates the sphinx indexes.

Sn0wCrack commented 7 years ago

Alright, adding the lastaudit datetime column (default NULL) to the Gallery table appears to get this working perfectly fine. But yes, unfortunately will take some time if you've have a lot of galleries. I'm going to leave this open until I double check that version updating is also working.

Sunako commented 7 years ago

👌

I've added the column, but I'll wait for you to confirm about version updating to try running it.

Sn0wCrack commented 7 years ago

Alright, I can confirm that Audit now works as intended. However it does require a second archive action to redownload the new gallery versions.

Sunako commented 7 years ago

You mean running the archive task again?

Sn0wCrack commented 7 years ago

Yeah, the audit task just adds the new galleries as "to be downloaded".

Sunako commented 7 years ago

Cool, I'll try it out later on. Going to sleep pretty soon.

Sn0wCrack commented 7 years ago

I have found an issue now actually with galleries that were removed due to being licensed content, such as comic X-EROS. So I'll reopen the issue, however you can manually index the changes currently.

Sunako commented 7 years ago

Yeah I was wondering about those galleries that are removed. Good thing I didn't try the audit task yet.

Sn0wCrack commented 7 years ago

Removed galleries just aren't returning any HTML at the moment for some reason, so it's something to look into further

Sn0wCrack commented 7 years ago

It seems ExHentai returns a 404 on these galleries which prevents cURL and things such as the Network Log in debug tools to fail to get a response. I'll just mark these like regular deleted galleries for now.