search

datagouv / validata.etalab.studio

4 stars 4 forks source link

Bug probable sur "validate" #8

Closed thbar closed 1 year ago

thbar commented 1 year ago

Hello,

J'ai hésité à ouvrir ce ticket sur https://git.opendatafrance.net/validata/validata-ui/-/issues mais ça pourrait être lié à la configuration nginx potentiellement, donc je commence ici.

J'ai la sensation qu'il y a peut-être un souci côté serveur sur le cas qui nous intéresse, car on a des erreurs avec plusieurs clients HTTP (Via curl ou paw, par exemple, on obtient des erreurs "server closed the connection too early, we expect a content-length of 331344" pour paw, ou une histoire de stream HTTP/2 non clôturé proprement).

Voir:

Quand on fait la requête suivante:

En navigateur (plus permissif), pas de souci.

Voir https://github.com/etalab/transport-site/issues/3088#issuecomment-1511685738 pour les exemples.

À votre disposition pour essayer de mieux comprendre, ça nous bloque sur une fonctionnalité de validation à la demande de notre côté (transport.data.gouv.fr).

Merci !!!

/cc @AntoineAugusti

AntoineAugusti commented 1 year ago

Démo avec curl en verbose

$ curl -v https://validata-api.app.etalab.studio/validate\?schema\=https://schema.data.gouv.fr/schemas/etalab/schema-comptage-mobilites-measure/latest/schema.json\&url\=https://github.com/etalab/transport-site/files/11246909/measures.csv
*   Trying 51.210.1.232...
* TCP_NODELAY set
* Connected to validata-api.app.etalab.studio (51.210.1.232) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-ECDSA-AES256-GCM-SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=validata-api.app.etalab.studio
*  start date: Feb 11 22:00:26 2023 GMT
*  expire date: May 12 22:00:25 2023 GMT
*  subjectAltName: host "validata-api.app.etalab.studio" matched cert's "validata-api.app.etalab.studio"
*  issuer: C=US; O=Let's Encrypt; CN=R3
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x7fb95500b400)
> GET /validate?schema=https://schema.data.gouv.fr/schemas/etalab/schema-comptage-mobilites-measure/latest/schema.json&url=https://github.com/etalab/transport-site/files/11246909/measures.csv HTTP/2
> Host: validata-api.app.etalab.studio
> User-Agent: curl/7.64.1
> Accept: */*
> 
* Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
< HTTP/2 200 
< server: nginx
< date: Tue, 18 Apr 2023 06:18:07 GMT
< content-type: application/json
< content-length: 331344
< vary: Accept-Encoding
< access-control-allow-origin: *
< strict-transport-security: max-age=15724800; includeSubdomains
< 
// BODY BODY BODY
* HTTP/2 stream 0 was not closed cleanly: INTERNAL_ERROR (err 2)
* stopped the pause stream!
* Connection #0 to host validata-api.app.etalab.studio left intact
curl: (92) HTTP/2 stream 0 was not closed cleanly: INTERNAL_ERROR (err 2)
05-29T00:00:00+02:00","55"],["353226380","CPTTEST2043","2022-05-29T00:00:00+02:00","2022-05-30T00:00:00+02:00","49"],["353226380","CPTTEST2043","2022-05-30T00:00:00+02:00","2022-05-31T00:00:00+02:00","37"],["353226380","CPTTEST2043","2022-05-31T00:00:00+02:00","2022-06-01T00:00:00+02:00","23"],["353226380","CPTTEST2043","2022-06-01T00:00:00+02:00","2022-06-02T00:00:00+02:00","17"],["353226380","CPTTEST2043","2022-06-02T00:00:00+02:00","2022-06-03T00:00:00+02:00","14"],["353226380","CPTTEST2043","2022-06-03T00:00:00+02:00","2022-06-04T00:00:00+02:00","28"],["353226380","CPTTEST2043","2022-06-04T00:00:00+02:00","2022-06-05T00:00:00+02:00","39"],["353226380","CPTTEST2043","2022-06-05T00:00:00+02:00","2022-06-06T00:00:00+02:00","35"],["353226380","CPTTEST2043","2022-06-06T00:00:00+02:00","2022-06-07T00:00:00+02:00","50"],["353226380","CPTTEST2043","2022-06-07T00:00:00+02:00","2022-06-08T00:00:00+02:00","26"],["353226380","CPTTEST2043","2022-06-08T00:00:00+02:00","2022-06-09T00:00:00+02:00","15"],["353226380","CPTTEST2043","2022-06-09T00:00:00+02:00","2022-06-10T00:00:00+02:00","22"],["353226380","CPTTEST2043","2022-06-10T00:00:00+02:00","2022-06-11T00:00:00+02:00","36"],["353226380","CPTTEST2043","2022-06-11T00:00:00+02:00","2022-06-12T00:00:00+02:00","50"],["353226380","CPTTEST2043","2022-06-12T00:00:00+02:00","2022-06-13T00:00:00+02:00","58"],["353226380","CPTTEST2043","2022-06-13T00:00:00+02:00","2022-06-14T00:00:00+02:00","34"],["353226380","CPTTEST2043","2022-06-14T00:00:00+02:00","2022-06-15T00:00:00+02:00","32"],["353226380","CPTTEST2043","2022-06-15T00:00:00+02:00","2022-06-16T00:00:00+02:00","22"],["353226380","CPTTEST2043","2022-06-16T00:00:00+02:00","2022-06-17T00:00:00+02:00","27"],["353226380","CPTTEST2043","2022-06-17T00:00:00+02:00","2022-06-18T00:00:00+02:00","14"],["353226380","CPTTEST2043","2022-06-18T00:00:00+02:00","2022-06-19T00:00:00+02:00","28"],["353226380","CPTTEST2043","2022-06-19T00:00:00+02:00","2022-06-20T00:00:00+02:00","40"],["353226380","CPTTEST2043","2022-06-20T00:00:00+02:00","2022-06-21T00:00:00+02:00","25"],["353226380","CPTTEST2043","2022-06-21T00:00:00+02:00","2022-06-22T00:00:00+02:00","32"],["353226380","CPTTEST2043","2022-06-22T00:00:00+02:00","2022-06-23T00:00:00+02:00","16"],["353226380","CPTTEST2043","2022-06-23T00:00:00+02:00","2022-06-24T00:00:00+02:00","21"],["353226380","CPTTEST2043","2022-06-24T00:00:00+02:00","2022-06-25T00:00:00+02:00","26"],["353226380","CPTTEST2043","2022-06-25T00:00:00+02:00","2022-06-26T00:00:00+02:00","20"],["353226380","CPTTEST2043","2022-06-26T00:00:00+02:00","2022-06-27T00:00:00+02:00","50"],["353226380","CPTTEST2043","2022-06-27T00:00:00+02:00","2022-06-28T00:00:00+02:00","32"],["353226380","CPTTEST2043","2022-06-28T00:00:00+02:00","2022-06-29T00:00:00+02:00","34"],["353226380","CPTTEST2043","2022-06-29T00:00:00+02:00","2022-06-30T00:00:00+02:00","24"],["353226380","CPTTEST2043","2022-06-30T00:00:00+02:00","2022-07-01T00:00:00+02:00","6"],["353226380","CPTTEST2043","2022-07-01T00:00:00+02:00","2022-07-02T00:00:00+02:00","37"],["353226380","CPTTEST2043","2022-07-02T00:00:00+02:00","2022-07-03T00:00:00+02:00","43"],["353226380","CPTTEST2043","2022-07-03T00:00:00+02:00","2022-07-04T00:00:00+02:00","82"],["353226380","CPTTEST2043","2022-07-04T00:00:00+02:00","2022-07-05T00:00:00+02:00","47"],["353226380","CPTTEST2043","2022-07-05T00:00:00+02:00","2022-07-06T00:00:00+02:00","32"],["353226380","CPTTEST2043","2022-07-06T00:00:00+02:00","2022-07-07T00:00:00+02:00","27"],["353226380","CPTTEST2043","2022-07-07T00:00:00+02:00","2022-07-08T00:00:00+02:00","37"],["353226380","CPTTEST2043","2022-07-08T00:00:00+02:00","2022-07-09T00:00:00+02:00","32"],["353226380","CPTTEST2043","2022-07-09T00:00:00+02:00","2022-07-10T00:00:00+02:00","57"],["353226380","CPTTEST2043","2022-07-10T00:00:00+02:00","2022-07-11T00:00:00+02:00","56"],["353226380","CPTTEST2043","2022-07-11T00:00:00+02:00","2022-
* Closing connection 0

Ce que je remarque, dans les headers de réponse :

L'erreur est toujours présente en forçant HTTP 1.1 curl -v --http1.1 https://validata-api.app.etalab.studio/validate\?schema\=https://schema.data.gouv.fr/schemas/etalab/schema-comptage-mobilites-measure/latest/schema.json\&url\=https://github.com/etalab/transport-site/files/11246909/measures.csv

geoffreyaldebert commented 1 year ago

merci pour le descriptif on va regarder avec @jordanguedj

thbar commented 1 year ago

@geoffreyaldebert j'ai l'impression que c'est réglé sur un test (mais peut-être que pas), pourrez-vous confirmer ? Merci !

thbar commented 1 year ago

@AntoineAugusti j'ai réutilisé le fichier fourni par Miryad, qui échouait, et ça semble marcher à présent c'est pour ça que je me pose la question.

AntoineAugusti commented 1 year ago

@thbar J'ai supprimé mon commentaire, t'as été rapide ⚡ J'ai constaté après que c'était résolu avec le fichier donné/en exécutant à nouveau les commandes curl

thbar commented 1 year ago

Yes, pas de souci ! J'attends la confirmation de l'équipe, peut-être qu'il y aura un seuil de taille à ne pas dépasser ou autre, ça sera bon de le savoir (ou peut-être que c'est juste globalement corrigé).

geoffreyaldebert commented 1 year ago

On démarre une prestation avec Multi cette semaine, si vous constatez à nouveau ce type d'erreur, n'hésitez pas à la signaler, on pourra regarder plus en détail. Je pars du principe que c'est réglé donc je close l'issue.