atviriduomenys / katalogas

Lietuvos atvirų duomenų katalogas (data.gov.lt).
14 stars 2 forks source link

CSV preview UnicodeDecodeError #782

Open sirex opened 1 year ago

sirex commented 1 year ago

https://data.gov.lt/datasets/1967/preview/10716/

Traceback (most recent call last):
  File "vitrina/datasets/views.py", line 416, in get
    data = list(csv.reader(rows, delimiter=";"))
  File "<frozen codecs>", line 322, in decode
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd0 in position 52: invalid continuation byte

XLSX failą bandoma atidaryti, kaip CSV failą. Bet taip pat, tame pačiame duomenų rinkinyje, atidarant ir CSV failą, gaunama ta pati unicode klaida.

Be to, preview atveju, neturėtu būti į atmintį užkraunamas visas failas, reikėtu nuskaityti tik kelias eilutes.

adp-atea commented 1 year ago

Uzdejau, kad rodytu penkias pirmas penkias eilutes

Faustasm commented 11 months ago

XLSX Testavimui: https://test.data.gov.lt/datasets/1138/#info https://test.data.gov.lt/datasets/1138/preview/8113/ 500 Klaida negrįžta, gaunami failo json, bet rodoma JSON.PARSE klaida. Matosi screenshote. CSV Testavimui: https://test.data.gov.lt/datasets/1604/#info https://test.data.gov.lt/datasets/1604/preview/14041/ 500 Klaida negrįžta, gaunami failo json.

Spaudžiant peržiūrėti prie csv failo https://test.data.gov.lt/datasets/1604/#info, modalinis atsidaro ir matosi informacija.

Spaudžiant peržiūrėti prie xlsx failo https://test.data.gov.lt/datasets/1138/#info, modalinis atsidaro, duomenys nesimato. Tikriausiai del minėto JSON.PARSE erroro

ss5
sirex commented 11 months ago

Klaida vis dar neišspręsta, atidarius https://test.data.gov.lt/datasets/2309/preview/13962/

Gaunu:

Traceback (most recent call last):
  File "django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
               ^^^^^^^^^^^^^^^^^^^^^
  File "django/core/handlers/base.py", line 181, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "django/views/generic/base.py", line 70, in view
    return self.dispatch(request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "django/views/generic/base.py", line 98, in dispatch
    return handler(request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "vitrina/datasets/views.py", line 449, in get
    data = list(csv.reader(rows, delimiter=";"))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen codecs>", line 322, in decode
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf8 in position 595: invalid start byte
Faustasm commented 9 months ago

Vis dar atsikartoja