search

metanorma / stepmod-utils

Tools for working on the STEPmod repository.
1 stars 0 forks source link

feat: create single command for srl migrate #182

Closed HassanAkbar closed 1 year ago

HassanAkbar commented 1 year ago

Create single command that migrates STEPmod repository into SRL

closes #160

HassanAkbar commented 1 year ago

@ronaldtse I have a question related to stepmod-srl-migrate-docs,

ronaldtse commented 1 year ago
  • In thedocuments/ folder do we only need the .adoc files? If yes then in stepmod2mn I didn't find any option to only generate the adoc files so, should I remove other files after the generation of documents?

What files are remaining?

HassanAkbar commented 1 year ago

@ronaldtse I've tested this on the iso-10303-stepmod-wg12 repo and the following files are also present inside the documents/{resources or modules}/ directory

TRThurman commented 1 year ago

What command initiates the process? Which file contains the list of documents to include in the SRL?

HassanAkbar commented 1 year ago

@TRThurman the documents folder is generated by the given command java -Xss5m -jar target/stepmod2mn-1.17.jar <Input-folder> --output documents

TRThurman commented 1 year ago

ok, so we will just tailor the input directory content before calling the command. Simple enough.

On Sep 28, 2023, at 8:40 AM, HassanAkbar @.***> wrote:

@TRThurman https://github.com/TRThurman the documents folder is generated by the given command java -Xss5m -jar target/stepmod2mn-1.17.jar --output documents

— Reply to this email directly, view it on GitHub https://github.com/metanorma/stepmod-utils/pull/182#issuecomment-1739218067, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAMMKVH6CDLQV62A4YG6LHLX4V46DANCNFSM6AAAAAA5HVXSVA. You are receiving this because you were mentioned.

ronaldtse commented 1 year ago

@ronaldtse I've tested this on the iso-10303-stepmod-wg12 repo and the following files are also present inside the documents/{resources or modules}/ directory

  • .svg

When you say .svg did you mean *.svg? Or an SVG with no name?

  • collection.sh

Shouldn't be there.

  • collection.yml

Shouldn't be there.

  • html_attachments.sh

Shouldn't be there.

  • schemas.yaml

Shouldn't be there.

Ping @Intelligent2013 on why these files are created.

Intelligent2013 commented 1 year ago

collection.sh calls html_attachments.sh for .htm attachments generation.

collection.yml is using in collection.sh:

bundle exec metanorma collection collection.yml -x xml,html,presentation -w iso10303-47

@opoudjis please confirm that collection.sh, collection.yml and html_attachments.sh are using in the metanorma files generation and we need them.

schemas.yaml is using in:

Note: collection.yml and schemas.yaml are unique for each document.

opoudjis commented 1 year ago

If you want attachments in the collection, and rendered as HTML as well as TXT at that, you need something to generate them. We went through that concern three years ago.

If you want collections to be processed, you need collection manifests.

If you want documents to be processed and then converted into collections with attachments, you need a script to do so.

"shouldn't be there" is wrong and disruptive. No attachments and no collection manifests and no collection scripts simply means no processing of the documents. @Intelligent2013 was about to cease generating those files, and that would have left you with no output but a single monolithic web page per part.

If you may have in mind some master process doing all those steps internal to stepmod2mn, and not generating config files, you should be architecting and communicating it. (What will actually happen is that stepmod2mn will ABSOLUTELY generate the config files, then run the compilation, and then delete the config files. In fact, that's how I'm recommending to @abunashir we process attachments within a single document in the general case; we're already doing pretty much for compiling documents in sectionsplit mode, with one page per clause. ISO-10303 needs both.)

However, given that

(a) I have spent a month and have only just got individual parts generating at all, (b) you want this all wrapped up by the end of the year, (c) Abu Nashir has only just started work on attachment compilation, (d) you are now onboarding someone new who has not had anything to do with the process to date, and is therefore unfamiliar with what is needed in it,

I suggest that disrupting a process we have not even consolidated yet is a bad idea. I have not even been able to generate two collections in tandem yet, and I will emphatically need access to these config files to do so.

Revisit this issue in a couple of months, when we've established a workflow for generating parts and collections of parts of the document, and after I have explained to the developers involved how the collections are generated. Right now, "shouldn't be there" is getting in the way of me doing my job.

HassanAkbar commented 1 year ago

@ronaldtse As pointed out by @opoudjis we need the generated files, so I think we should not remove them for now, we can revisit this issue later.