validation fails on a legit species!

[mkoo]

Hasan has tried to upload data with Taricha species but validation sends error that names are not valid which is not true. Appears to be a ADP validation failure. XLS posted here to troubleshoot! ERROR msg: Taxonomy Error: There was a taxon error in your file. 'Taricha' isn't a valid AmphibiaWeb genus (checked 0 genera), nor is 'taricha rivularis' a recognized synonym. The error occured while we were checking taxon "Taricha rivularis", which occurs at rows 1,2,3,4,60.... We stopped validation at that point. Please correct taxonomy issues and try uploading again. If you're confused by this message, please check our documentation . attempt was on Chrome/Win

[tigerhawkvok]

The problem comes because the server hasn't been able to fetch the AmphibiaWeb list (it's been getting an empty result).

This old file works, for example:

aweb_list.txt

The new files are all length zero.

@JoyceGross has there been a permissions change for file_get_contents? It's fetching only 0-byte results.

There are several things for me to check here as to why it's failing now when it wasn't before, but as a stopgap I've added an explicit check to ensure that it fetches more than 0 bytes before overwriting a list file with data in it.

[JoyceGross]

The file is still there, with all the names; nothing has changed with the file itself.

https://amphibiaweb.org/amphib_names.txt

What has changed is mod_security is running on louise now. It looks as if mod_security isn't happy with the way the file is being requested. See lines from the modsec_audit.log file below -- "Request Missing a User Agent Header"

I'll look into disabling that rule but I don't know how to test that it's working for you, so someone else will have to do that.

Joyce

--894f551f-A-- [12/Dec/2017:08:02:22 --0800] Wi-9joAg1Z0AACvmMjsAAAAO 128.32.213.156 42804 128.32.213.157 443 --894f551f-B-- GET /amphib_names.txt HTTP/1.0 Host: amphibiaweb.org

--894f551f-F-- HTTP/1.1 403 Forbidden Content-Length: 300 Connection: close Content-Type: text/html; charset=iso-8859-1

--894f551f-E--

--894f551f-H-- Message: Access denied with code 403 (phase 2). Operator EQ matched 0 at REQUEST_HEADERS. [file "/etc/httpd/modsecurity.d/activated_rules/modsecurity_crs_21_protocol_anomalies.conf"] [line "67"] [id "960009"] [rev "1"] [msg "Request Missing a User Agent Header"] [severity "NOTICE"] [ver "OWASP_CRS/2.2.6"] [maturity "9"] [accuracy "9"] [tag "OWASP_CRS/PROTOCOL_VIOLATION/MISSING_HEADER_UA"] [tag "WASCTC/WASC-21"] [tag "OWASP_TOP_10/A7"] [tag "PCI/6.5.10"] Action: Intercepted (phase 2) Stopwatch: 1513094542732796 40057 (- - -) Stopwatch2: 1513094542732796 40057; combined=326, p1=151, p2=44, p3=0, p4=0, p5=74, sr=35, sw=0, l=0, gc=57 Response-Body-Transformed: Dechunked Producer: ModSecurity for Apache/2.7.3 (http://www.modsecurity.org/); OWASP_CRS/2.2.6. Server: Apache/2.2.15 (CentOS) Engine-Mode: "ENABLED"

--894f551f-Z--

[JoyceGross]

I disabled rule 960009 for the file amphib_names.txt. Please let me know if that solves the issue or not. Would not surprise me if there's going to be another mod_security issue.

[tigerhawkvok]

@JoyceGross that seems to have resolved it, messages on the API are now no longer reporting read failures and a fresh age for the aweb list (small value for aweb_list_age).

https://amphibiandisease.org/api.php?action=validate&genus=lithobates&species=catesbeianus

{"status":true,"args_provided":{"action":"validate","genus":"lithobates","species":"catesbeianus"},"notices":{"0":"Your entry 'lithobates catesbeianus' was a synonym in the AmphibiaWeb database. It was automatically converted to the canonical taxon."},"aweb_list_age":20,"aweb_list_max_age":86400,"original_taxon":"lithobates catesbeianus","validated_taxon":{"order":"Anura","family":"Ranidae","subfamily":"","genus":"Rana","subgenus":"Aquarana","species":"catesbeiana","common_name":{"0":"Bullfrog","1":"American Bullfrog"},"gaa_name":"Lithobates catesbeianus","synonymies":"","itis_names":"Lithobates catesbeianus","iucn":"Least Concern (LC)","isocc":{"0":"AR","1":"BE","2":"BR","3":"CA","4":"CN","5":"CO","6":"CU","7":"DO","8":"EC","9":"FR","10":"DE","11":"GR","12":"HT","13":"ID","14":"IT","15":"JM","16":"JP","17":"MY","18":"MX","19":"PE","20":"PH","21":"PR","22":"SG","23":"ES","24":"TW","25":"TH","26":"US","27":"UY","28":"VE"},"intro_isocc":{"0":"TW","1":"PR","2":"JP","3":"JM","4":"IT","5":"HT","6":"DO","7":"CU","8":"CN","9":"FR","10":"CO","11":"BR","12":"EC","13":"AR","14":"UY","15":"VE","16":"GR","17":"BE","18":"DE","19":"ID","20":"MY","21":"PE","22":"PH","23":"SG","24":"ES","25":"TH"},"aweb_uid":"4999","uri_or_guid":"https:\/\/amphibiaweb.org\/species\/4999","taxon_notes_public":{"0":"This species was placed in the genus Lithobates by Frost et al. (2006). However","1":"Yuan et al. (2016","2":"Systematic Biology","3":"doi: 10.1093\/sysbio\/syw055) showed that  this action created problems of paraphyly in other genera. Yuan et al. (2016)  recognized subgenera within Rana for the major traditional species groups","4":"with  Lithobates used as the subgenus for the Rana palmipes group. AmphibiaWeb  recommends the optional use of these subgenera to refer to these major species  groups","5":"with names written as Rana (Aquarana) catesbeiana","6":"for example."}},"execution_time":176.28002166748}