Tree figure uploads fail

[Ksepka]

This morning I tried to upload a tree figure from my work computer and was unable to do so. Error message below. Our network here is very wonky and I have never encountered this error on my laptop which I usually use for this work. Just noting here - I will retest at home and see if the problem goes away but was interested in why it arise in case it effects others.

Error in query: INSERT INTO calibrations SET CalibrationID = '' ,NodeName = '' ,HigherTaxon = '' ,MinAge = '' ,MinAgeExplanation = '' ,MaxAge = '' ,MaxAgeExplanation = '' ,NodePub = '' ,PublicationStatus = '1' ,CalibrationQuality = '' ,PrimaryLinkedFossilID = '' ,AdminComments = '' ON DUPLICATE KEY UPDATE CalibrationID = '' ,NodeName = '' ,HigherTaxon = '' ,MinAge = '' ,MinAgeExplanation = '' ,MaxAge = '' ,MaxAgeExplanation = '' ,NodePub = '' ,PublicationStatus = '1' ,CalibrationQuality = '' ,PrimaryLinkedFossilID = '' ,AdminComments = '' |Cannot add or update a child row: a foreign key constraint fails (FossilCalibration.calibrations, CONSTRAINT calibrations_ibfk_2 FOREIGN KEY (CalibrationQuality) REFERENCES L_CalibrationQuality (QualityID))

[jimallman]

Hm, this looks like we're trying to create (or update) a calibration with no ID and all empty fields. I'm not surprised it's an error! I'll take a look at the logs.

Please confirm: This was on the proper site (fossilcalibrations.org) and not the old dev site (fossils.ibang.com), right? Which calibration?

[Ksepka]

It is the fossil calibrations.org. I was attempting this with Marsupialia but got the same issue with other calibrations. It seems to only happen when I try to upload a tree - I am suspicious it is local to my work computer. Can test tonight.

[Ksepka]

I tested this and I was also unable to upload a tree for the calibration "Laurales". This particular file was rather large so I tried reducing the size and it then worked fine. I assume there is a file size limit for tree uploads and it appears to be less than 1MB. Is this correct?

[jimallman]

The default for PHP is 2MB, but this is easy to change. Do you have a maximum size in mind?

@dleehr, these links discuss how to relax upload limits using php.ini or per-directory .htaccess files:

http://stackoverflow.com/questions/2184513/php-change-the-maximum-upload-file-size http://www.cyberciti.biz/faq/linux-unix-apache-increase-php-upload-limit/

[pdpolly]

P. David Polly Professor, Geological Sciences Adjunct, Anthropology and Biology Indiana University 1001 E. 10th Street Bloomington, IN 47405-1405 pdpolly@indiana.edu +1 (812) 855-7994 http://mypage.iu.edu/~pdpolly/

On 15 Dec 2014, at 1:54 AM, Jim Allman notifications@github.com wrote:

The default for PHP is 2MB, but this is easy to change. Do you have a maximum size in mind?

@dleehr https://github.com/dleehr, these links discuss how to relax upload limits using php.ini or per-directory .htaccess files:

http://stackoverflow.com/questions/2184513/php-change-the-maximum-upload-file-size http://stackoverflow.com/questions/2184513/php-change-the-maximum-upload-file-size http://www.cyberciti.biz/faq/linux-unix-apache-increase-php-upload-limit/ http://www.cyberciti.biz/faq/linux-unix-apache-increase-php-upload-limit/ — Reply to this email directly or view it on GitHub https://github.com/NESCent/FossilCalibrations/issues/63#issuecomment-66955887.

[dleehr]

Good catch all, will update the site and the documentation.

[dleehr]

@Ksepka I've raised the upload limits, let me know if this is working now.

[jimallman]

Once we know what a reasonable limit should be (8MB? 40MB?), we should add this to step 1 (PHP configuration) of the wiki's Installation page.

[pdpolly]

Are these tree files (e.g., Nexus or Newick format trees) or graphics figures showing the phylogenetic tree? I'd go for files that are 72 dpi for screen reading, about 5-8 inches in width, jpeg format. Those ought to be less than 8MB, probably quite a bit less.

On 15 Dec 2014, at 10:17 AM, Jim Allman notifications@github.com wrote:

Once we know what a reasonable limit should be (8MB? 40MB?), we should add this to step 1 (PHP configuration) of the wiki's Installation page https://github.com/NESCent/FossilCalibrations/wiki/Installation.

— Reply to this email directly or view it on GitHub https://github.com/NESCent/FossilCalibrations/issues/63#issuecomment-67008496.

[Ksepka]

David these are graphics (jpg usually).

This morning I attempted to upload some files and the limit seems to still be >2MB. In the test calibration "dan", I was unable to upload a 700KB jpeg. I resized it to 350KB and it uploaded just fine. Both versions were just the author TIFF opened and reserved at different sizes using Preview. The issue seems to be file size rather (KB) rather than image proportions (e.g., it does not have to be a certain DPI or dimensions as far as I can tell). A limit of 2MB should be fine I would think based on how these look at scale.

[jparham]

I would say give some high limit so as to avoid any issue down the line. Could save time if someone has a 2.1 MB file, they don't have to resize. What is the cost with having a 4, 5, 10 MB limit?

[dleehr]

I raised the php max upload/post (200MB) and memory size (1024MB), but this still fails. The failure is in update_calibration.php:L321

So the file may be received by php correctly, but not encoded for the database storage. I'll investigate.

[jimallman]

Ah, of course it could also be size restricted in the database. I'm checking the MySQL schema now to see how big this field is..

UPDATE: The field publicationimages.image (used for these tree images) is of type MEDIUMBLOB, which is 16 MB. If we really need more than this for a web graphic, I think we should take a closer look at the file formats and compression used!

[dleehr]

Yeah I was just looking at that too. Also, the data is base64 encoded, which means the limit is actually about 12MB. Right now I'm trying to test with a 3MB image that is about 4MB in base64 and it still fails, but the failure is now after the insert, so I'm making progress.

[dleehr]

Actually, I was able to fix this. I was testing on a local VM but the last line of update_calibration.php redirects to the wrong url

[dleehr]

I updated the server and the wiki with some higher limits, and my 3MB upload succeeded.

https://github.com/NESCent/FossilCalibrations/wiki/Installation/885668a1e5e057f2fbcd8f914fea6d092130888b

[Ksepka]

Thanks all!