Any plans to implement converter to html from .doc version of spec?

[slashTPA]

Hi,

while retrieving .doc file works well for latest release (e.g. rel-16) it looks like that no converter towards pdf or html is implemented. Hence html is only available for older versions (e.g. rel-15).

Do you have any plans to implement converter to html from .doc version of spec? Do you have any more detailed docu that might enlighten me to contribute to this project?

Cheers/T

[CoRfr]

Hi @slashTPA , not really any plans. I remember that I tried at the time but couildn't find a good solution to script the conversion of doc to pdf/html. That's why so far I was relying on the ETSI pdf, but it seems that sometimes they are not available, so ideally we should find a way to export the doc.

I had a quick try with libreoffice ( https://libreofficehelp.com/batch-convert-writer-documents-pdf-libreoffice/ ):

lowriter -env:UserInstallation=file:///$HOME/.libreoffice-headless/ --headless --convert-to pdf ~/Downloads/36216-f00.doc

That seems to produce a pdf that looks okay (except for the front page). So I would say it's possible to convert it if we find lowriter for instance, and we should be able to easily package that in the Docker container (although it will probably make the image much heavier - but that's probably ok in our case).

[slashTPA]

Hi @CoRfr,

ETSI PDF becomes available just (somewhen) after the plenary meetings. As of now, that limits the HTML usage to Rel-15 specs, while Rel-16 is available only at 3gpp site as DOC.

I had a look as well into this: $ soffice -env:UserInstallation=file:///$HOME/.libreoffice-headless/ --convert-to pdf ~/Downloads/33501-g20.doc and besides frontpage and some characters inside pictures/message flows - it seems better than nothing. It might be worth to implement for then a check if that certain version exists on ETSI already, and if not - fetch the doc and convert to pdf first. But then a cleanup would be needed once the PDF becomes available... Not sure if that is worth the efforts (?) - it sort of destroys the lighwight approach...

another option might be https://pandoc.org/ as that seems to convert straight into HTML. Will check that later on.

[CoRfr]

It means that we would need to track the PDFs that have been converted from DOC, and periodically check if there is an ETSI version available.

There is already a cron in the docker container to periodically (weekly) sync from 3gpp website so it might not be that difficult to add that ETSI check. For the doc => html path, it might be best to keep only one path, through pdf, using pdf2htmlEX ? I mean that could be optimized too, but it would make things more complex.

I tried pandoc but I ran into:

Unknown input format doc
Pandoc can convert from DOCX, but not from DOC.
Try using Word to save your DOC file as DOCX, and convert that with pandoc.

One thing that I regret is that at the moment the file processing is taken care of in the frontend. I mean there should be some sort of pool of worker to process the files. And the files should probably be hosted on a S3 bucket or something like that.

[slashTPA]

I agree, for the doc => html path, having only one path is the better approach. And yes, pandoc unfortunately requires docx, I wasn't aware of that limitation. That would only let the soffice as remaining option.... Actually I do not see an issue with the processing in the frontend - it doesn't eat up so much time (?) the point of hosting would be a valid option for improvement. Do you have any plans there?

[CoRfr]

Not really any plans either - my contributions to this project are now very limited as I do not use 3GPP documents on a daily basis like I was at the time were I wrote this. But I'll happily take care of integrations and merges of PRs, so if you want to give it a shot that will be a good thing I think.

For the frontend processing, it's true that it should be a one shot operation per document, so not a big deal overall.

[slashTPA]

OK, cool. Currently I'm a bit limited in time, but I'll try to squeeze that in ;-). May I ask, do you have any additional docu on the entire project that could be shared? (idea behind db-structure... )