search

GSA / data.gov

Main repository for the data.gov service
https://data.gov
Other
595 stars 92 forks source link

Datastore upload JavaScript fails to load in staging #747

Closed adborden closed 5 years ago

adborden commented 5 years ago

The JavaScript code for the inventory datastore upload fails to load, connection reset error.

curl -v 'https://inventory-datagov.dev-ocsit.bsp.gsa.gov/fanstatic/base/:version:2019-04-02T02:22:38.91/:bundle:plugins/jquery.inherit.min.js;plugins/jquery.proxy-all.min.js;plugins/jquery.url-helpers.min.js;plugins/jquery.date-helpers.min.js;plugins/jquery.slug.min.js;plugins/jquery.slug-preview.min.js;plugins/jquery.truncator.min.js;plugins/jquery.masonry.min.js;plugins/jquery.form-warning.min.js;sandbox.min.js;module.min.js;pubsub.min.js;client.min.js;notify.min.js;i18n.min.js;main.min.js;modules/select-switch.min.js;modules/slug-preview.min.js;modules/basic-form.min.js;modules/confirm-action.min.js;modules/api-info.min.js;modules/autocomplete.min.js;modules/custom-fields.min.js;modules/related-item.min.js;modules/data-viewer.min.js;modules/table-selectable-rows.min.js;modules/resource-form.min.js;modules/resource-upload-field.min.js;modules/resource-reorder.min.js;modules/resource-view-reorder.min.js;modules/follow.min.js;modules/activity-stream.min.js;modules/dashboard.min.js;modules/resource-view-embed.min.js;view-filters.min.js;modules/resource-view-filters-form.min.js;modules/resource-view-filters.min.js;modules/table-toggle-more.min.js;modules/dataset-visibility.min.js;modules/media-grid.min.js;modules/image-upload.min.js'

This appears to be a WAF or firewall issue because the request does not appear in the apache logs.

How to reproduce

  1. https://inventory-datagov.dev-ocsit.bsp.gsa.gov/dataset/new_resource/per-diem-reimbursement-rates
  2. Set Resource to "Upload a file"

Expected behavior

JavaScript loads correctly and Upload button appears.

Actual behavior

JavaScript fails to load with 'Failed to load resource: net::ERR_CONNECTION_RESET' and Upload button does not appear.

adborden commented 5 years ago

Opened RITM0603488 to help debug.

adborden commented 5 years ago

It's unclear why, but the URL is now returning 200 as expected.

adborden commented 5 years ago

This is now happening on production and not staging...

$ curl -v 'https://inventory.data.gov/fanstatic/base/:version:2019-05-29T18:35:52.25/:bundle:plugins/jquery.inherit.min.js;plugins/jquery.proxy-all.min.js;plugins/jquery.url-helpers.min.js;plugins/jquery.date-helpers.min.js;plugins/jquery.slug.min.js;plugins/jquery.slug-preview.min.js;plugins/jquery.truncator.min.js;plugins/jquery.masonry.min.js;plugins/jquery.form-warning.min.js;sandbox.min.js;module.min.js;pubsub.min.js;client.min.js;notify.min.js;i18n.min.js;main.min.js;modules/select-switch.min.js;modules/slug-preview.min.js;modules/basic-form.min.js;modules/confirm-action.min.js;modules/api-info.min.js;modules/autocomplete.min.js;modules/custom-fields.min.js;modules/related-item.min.js;modules/data-viewer.min.js;modules/table-selectable-rows.min.js;modules/resource-form.min.js;modules/resource-upload-field.min.js;modules/resource-reorder.min.js;modules/resource-view-reorder.min.js;modules/follow.min.js;modules/activity-stream.min.js;modules/dashboard.min.js;modules/resource-view-embed.min.js;view-filters.min.js;modules/resource-view-filters-form.min.js;modules/resource-view-filters.min.js;modules/table-toggle-more.min.js;modules/dataset-visibility.min.js;modules/media-grid.min.js;modules/image-upload.min.js'
adborden commented 5 years ago

Back in April, SecOps confirmed that the length limit is 3072 and would appear in their logs if the URL was blocked due to line length.

adborden commented 5 years ago

SecOps doesn't see anything in their logs. Based on packet capture on the client, the connection is being reset at the NetScaler instance.

I tried disabling modsecurity to make sure this wasn't an issue on our side, but it had no effect.

adborden commented 5 years ago

SecOps got back to us and tweaked the TrendMicro configuration. This is working in production and staging now.