search

rust-ml / book

The Rust Machine Learning Book
https://rust-ml.github.io/book/
Apache License 2.0
196 stars 20 forks source link

Prepare a docker image for build mdbook-scientific #8

Closed liufuyang closed 3 years ago

liufuyang commented 3 years ago

related to https://github.com/rust-ml/book/issues/5#issuecomment-801730275

Basic idea is to have a docker image that contains working programs such as mdbook and mdbook-scientific.

WIP, need to install bib2xhtml in the build image (alpine linux)

For now at least mdbook and mdbook-scientific are packed into the docker image.

docker run --rm -it -v $(pwd):/github/workspace liufuyang/mdbook-scientific:0.3.7 build

2021-03-23 08:29:50 [INFO] (mdbook::book): Book building has started
thread 'main' panicked at 'Could not spawn bib2xhtml: Os { code: 2, kind: NotFound, message: "No such file or directory" }', src/fragments.rs:250:10
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
2021-03-23 08:29:50 [ERROR] (mdbook::utils): Error: The preprocessor exited unsuccessfully

We can switch the docker image to your organization's image base, and perhaps move docker file to another repo, if that is what you think a better thing to do. Now I would like to illustrate the CI idea via this PR and perhaps you can comment on it.

Also, in order for me to test this, do you mind create a master-test-build branch from current master? Thank you :)

bytesnake commented 3 years ago

I added a master-test-build branch for you, also let me know if you need anything else (I can also grant more access to the repository if you need) you can find the bib2xhtml script here

liufuyang commented 3 years ago

Thanks. I tried to add that bib2xhtml.pl file and now I see error like this: perhaps means I need some gnuplot or?

/github/workspace # mdbook build
2021-04-01 10:00:57 [INFO] (mdbook::book): Book building has started
Block empty, but file `hyperplane` was not found!
Error in chapter 1. BinaryNotFound(Error { inner: CannotFindBinaryPath })
2021-04-01 10:00:57 [ERROR] (mdbook::utils): Error: The preprocessor exited unsuccessfully

About more access, if I later I can push some changes directly to master-test-build to test the build then it would be nice. Let's see, maybe the first build can work out of the box :P

liufuyang commented 3 years ago

I tried to locally install gnuplot and running a build, it also not working and showing the same error above. What is that hyperlane file about?

liufuyang commented 3 years ago

Tried to install texlive package in the alpine image now I see this error:

2021-04-01 10:18:00 [INFO] (mdbook::book): Book building has started
InvalidBibliography("This is BibTeX, Version 0.99d (TeX Live 2020/Alpine Linux)\nThe top-level auxiliary file: bib10.aux\nI couldn\'t open style file html-a.bst\n---line 2 of file bib10.aux\n : \\bibstyle{html-a\n :                 }\nI\'m skipping whatever remains of this command\nI found no style file---while reading file bib10.aux\n(There were 2 error messages)\n")
2021-04-01 10:18:00 [ERROR] (mdbook::utils): Error: The preprocessor exited unsuccessfully

No I have no clue what to do :)

liufuyang commented 3 years ago

Now I could get a bit further by doing that ./gen-bst.pl operation from bib2xhtml repo, and using a ubuntu image and installing texlive-latex-extra solves some other issue.

But now I get back this hyperplane problem... Not sure why. Seem it was gone previously with alpine linux and gnuplot + texlive installed

2021-04-01 13:35:20 [INFO] (mdbook::book): Book building has started
Block empty, but file `hyperplane` was not found!
Block empty, but file `hyperplane` was not found!
Error in chapter : InvalidReference("could not find reference to `hyperplane` in line 5")
2021-04-01 13:35:20 [ERROR] (mdbook::utils): Error: The preprocessor exited unsuccessfully
bytesnake commented 3 years ago

it seems that the hyperplane.tex file is missing, a statement without any body, e.g. $ref:fig:hyperplane$ means that he should look for a file called hyperplane.tex

bytesnake commented 3 years ago

Thanks. I tried to add that bib2xhtml.pl file and now I see error like this: perhaps means I need some gnuplot or?

/github/workspace # mdbook build
2021-04-01 10:00:57 [INFO] (mdbook::book): Book building has started
Block empty, but file `hyperplane` was not found!
Error in chapter 1. BinaryNotFound(Error { inner: CannotFindBinaryPath })
2021-04-01 10:00:57 [ERROR] (mdbook::utils): Error: The preprocessor exited unsuccessfully

About more access, if I later I can push some changes directly to master-test-build to test the build then it would be nice. Let's see, maybe the first build can work out of the box :P

I will grant you access then to the branch

liufuyang commented 3 years ago

@bytesnake interesting, I can see there is a hyperplane.tex in the src file. Are you sure the mdbook-scientific currently can working correctly or it is still some unstable version? I tried to remove all the parts that related to hyperplane in chapter_1.md then it can build the equations and plots.

liufuyang commented 3 years ago

I guess we can merge this and give it a try to see whether the gh-pages branch got build.

One issue or bug is that when I run mdbook serve (actually via docker run --rm -v $(pwd):/github/workspace --name mdbook -p 3000:3000 liufuyang/mdbook-scientific:0.3.7 serve -n 0.0.0.0) then after updating some equations, the CPU usage is high and it keeps on reporting log:

2021-04-01 16:10:31 [INFO] (mdbook::cmd::serve): Files changed: ["/github/workspace/src/assets/439706c949677821f5e69104.svg", "/github/workspace/src/assets/d36d2f91a62a9e32d4bdb7d7.svg", "/github/workspace/src/assets/e9ea986c569b3a41be529169.svg", "/github/workspace/src/assets/de566924d07db43d343b3454.svg"]
2021-04-01 16:10:31 [INFO] (mdbook::cmd::serve): Building book...
2021-04-01 16:10:31 [INFO] (mdbook::book): Book building has started
2021-04-01 16:10:31 [INFO] (mdbook::book): Running the html backend
2021-04-01 16:10:32 [INFO] (mdbook::cmd::serve): Files changed: ["/github/workspace/src/assets/439706c949677821f5e69104.svg", "/github/workspace/src/assets/d36d2f91a62a9e32d4bdb7d7.svg", "/github/workspace/src/assets/e9ea986c569b3a41be529169.svg", "/github/workspace/src/assets/de566924d07db43d343b3454.svg"]
2021-04-01 16:10:32 [INFO] (mdbook::cmd::serve): Building book...

Maybe this can be fixed later on...

liufuyang commented 3 years ago

Note that I made this PR to merge into rust-ml:master-test-build so it's okay to merge now for test purpose. Otherwise if you have some hints on how to fix the hyperplane issue, please let me know.

liufuyang commented 3 years ago

Basically right now you can try to build the site via the docker image I prepared from the Dockerfile here.

So if you are at a master branch locally, you can try setting bib2xhtml = "/github/bib2xhtml" in the book.toml and use this command to make a build

docker run --rm -v $(pwd):/github/workspace liufuyang/mdbook-scientific:0.3.7 build

Then you will see the hyperplane error...

liufuyang commented 3 years ago

@bytesnake have you seen some of my questions above? I think I encountered some error while building the book with the scientific plugins. Are we sure the current master of this book can be build by the mdbook with scientific plugin on your side? Sorry I missed an invitation for you to invite to join as a collaborator. It could be nice if you can help send again? :)

bytesnake commented 3 years ago

sorry bit busy atm :sweat_smile:

@bytesnake interesting, I can see there is a hyperplane.tex in the src file. Are you sure the mdbook-scientific currently can working correctly or it is still some unstable version? I tried to remove all the parts that related to hyperplane in chapter_1.md then it can build the equations and plots.

it worked on my side, it should be able to locate the hyperplane.tex in src/. If you want, I can try to run the docker integration, but have to install docker first to my computer and this will take some time

I guess we can merge this and give it a try to see whether the gh-pages branch got build.

One issue or bug is that when I run mdbook serve (actually via docker run --rm -v $(pwd):/github/workspace --name mdbook -p 3000:3000 liufuyang/mdbook-scientific:0.3.7 serve -n 0.0.0.0) then after updating some equations, the CPU usage is high and it keeps on reporting log:

2021-04-01 16:10:31 [INFO] (mdbook::cmd::serve): Files changed: ["/github/workspace/src/assets/439706c949677821f5e69104.svg", "/github/workspace/src/assets/d36d2f91a62a9e32d4bdb7d7.svg", "/github/workspace/src/assets/e9ea986c569b3a41be529169.svg", "/github/workspace/src/assets/de566924d07db43d343b3454.svg"]
2021-04-01 16:10:31 [INFO] (mdbook::cmd::serve): Building book...
2021-04-01 16:10:31 [INFO] (mdbook::book): Book building has started
2021-04-01 16:10:31 [INFO] (mdbook::book): Running the html backend
2021-04-01 16:10:32 [INFO] (mdbook::cmd::serve): Files changed: ["/github/workspace/src/assets/439706c949677821f5e69104.svg", "/github/workspace/src/assets/d36d2f91a62a9e32d4bdb7d7.svg", "/github/workspace/src/assets/e9ea986c569b3a41be529169.svg", "/github/workspace/src/assets/de566924d07db43d343b3454.svg"]
2021-04-01 16:10:32 [INFO] (mdbook::cmd::serve): Building book...

Maybe this can be fixed later on...

yeah there is a bug in the serve mode, which will automatically retrigger itself. I didn't continued working on that because building works.

Note that I made this PR to merge into rust-ml:master-test-build so it's okay to merge now for test purpose. Otherwise if you have some hints on how to fix the hyperplane issue, please let me know.

perfect :+1:

@bytesnake have you seen some of my questions above? I think I encountered some error while building the book with the scientific plugins. Are we sure the current master of this book can be build by the mdbook with scientific plugin on your side? Sorry I missed an invitation for you to invite to join as a collaborator. It could be nice if you can help send again? :)

have re-sent the invitation

liufuyang commented 3 years ago

I see. Thanks for the info. I will firstly test whether the github action thing works or not. If that can work, then the left over is to somehow make the docker image can build the current master code.

liufuyang commented 3 years ago

I will merge this now and see weather this triggers a build for us or not :)