procesar los datos para lo de SDS

[omazapa]

• [x] tomar los de kahi
• [x] hacer los parsers
• [ ] corregir los notebooks
• [ ] sacar los datos
• [ ] probar en docker (con Omar) errores reportados abajo
• [x] anonimizar los datos para ponerlos en el docker
• [x] crear los indices para todas las entidades (Omar)

[omazapa]

• [ ] Error 1 File "/usr/local/lib/python3.8/dist-packages/sds/plugins/InstitutionsApp.py", line 703, in app_institutions sds_backend | production=self.get_production_by_type(idx=idx, sds_backend | File "/usr/local/lib/python3.8/dist-packages/sds/plugins/InstitutionsApp.py", line 618, in get_production_by_type sds_backend | name=sub["names"][0]["name"] sds_backend | KeyError: 'names'

https://github.com/colav/sds-backend/blob/main/sds/plugins/InstitutionsApp.py#L618

En aff no tiene ese estructura

• [ ] Error 2 sds_backend | File "/usr/local/lib/python3.8/dist-packages/sds/plugins/AuthorsApp.py", line 634, in app_authors sds_backend | production=self.get_production_by_type(idx=idx, sds_backend | File "/usr/local/lib/python3.8/dist-packages/sds/plugins/AuthorsApp.py", line 533, in get_production_by_type sds_backend | name=sub["names"][0]["name"]

https://github.com/colav/sds-backend/blob/main/sds/plugins/AuthorsApp.py#L530

• [ ] Error 3 ds_backend | File "/usr/local/lib/python3.8/dist-packages/sds/plugins/SubjectsApp.py", line 983, in app_subjects sds_backend | authors=self.get_authors(idx,page,max_results,start_year,end_year,groups,institutions,sort,"descending") sds_backend | File "/usr/local/lib/python3.8/dist-packages/sds/plugins/SubjectsApp.py", line 897, in get_authors sds_backend | name=sub["names"][0]["name"] sds_backend | KeyError: 'names'

https://github.com/colav/sds-backend/blob/main/sds/plugins/SubjectsApp.py#L894

• [ ] Error 4 sds_backend | File "/usr/local/lib/python3.8/dist-packages/sds/plugins/CompendiumApp.py", line 756, in app_compendium sds_backend | authors=self.get_authors() sds_backend | File "/usr/local/lib/python3.8/dist-packages/sds/plugins/CompendiumApp.py", line 476, in get_authors sds_backend | if deg["date"]>max_time:

No tenemos esos datos, tomar de los viejos y cruzar con COD_RH ya que eso debe venir de cvlac.

• [ ] Error 5 http://172.22.0.4/sds_api/app/search?data=subjects&keywords=cirugia&max=10&page=1&sort=citations

Busqueada por tema, ej: cirugia.

• [ ] Error 5 dandole click a un artículo sds_backend | File "/usr/local/lib/python3.8/dist-packages/sds/plugins/DocumentsApp.py", line 149, in app_document sds_backend | info = self.get_info(idx) sds_backend | File "/usr/local/lib/python3.8/dist-packages/sds/plugins/DocumentsApp.py", line 77, in get_info sds_backend | for typ in author["types"]: sds_backend | KeyError: 'types'

https://github.com/colav/sds-backend/blob/main/sds/plugins/DocumentsApp.py#L77

• [ ] Error 6 en coautorias de grupos (perfil de grupo)

http://172.22.0.4/app/groups?id=665e0b60aa7c2077adf683d3

sds_backend | File "/usr/local/lib/python3.8/dist-packages/sds/plugins/GroupsApp.py", line 693, in app_groups sds_backend | coauthors=self.get_coauthors(idx,start_year,end_year) sds_backend | File "/usr/local/lib/python3.8/dist-packages/sds/plugins/GroupsApp.py", line 313, in get_coauthors sds_backend | for reg in self.colav_db["works"].aggregate(pipeline,allowDiskUse=True): sds_backend | File "/usr/local/lib/python3.8/dist-packages/pymongo/collection.py", line 2392, in aggregate sds_backend | return self._aggregate(pipeline, sds_backend | File "/usr/local/lib/python3.8/dist-packages/pymongo/collection.py", line 2293, in _aggregate sds_backend | result = sock_info.command( sds_backend | File "/usr/local/lib/python3.8/dist-packages/pymongo/pool.py", line 570, in command sds_backend | return command(self.sock, dbname, spec, slave_ok, sds_backend | File "/usr/local/lib/python3.8/dist-packages/pymongo/network.py", line 148, in command sds_backend | helpers._check_command_response( sds_backend | File "/usr/local/lib/python3.8/dist-packages/pymongo/helpers.py", line 155, in _check_command_response sds_backend | raise OperationFailure(msg % errmsg, code, response) sds_backend | pymongo.errors.OperationFailure: Total size of documents in affiliations matching pipeline { $match: { $and: [ { _id: { $eq: ObjectId('665e0a9caa7c2077adf62c7f') } }, {} ] } } exceeds maximum document size

No es un problems de datos el backend saca un error por exceder el tamaño del document en el aggregate