databricks / spark-deep-learning

Deep Learning Pipelines for Apache Spark
https://databricks.github.io/spark-deep-learning
Apache License 2.0
1.99k stars 494 forks source link

[ML-5700] Update release script to build and push docs. #176

Closed MrBago closed 5 years ago

MrBago commented 5 years ago

This PR adds doc building to the release script. The script will checking the release tag and build the docs in docker. It will then push the docs branch, eg "gh-pages-1.4.0" upstream & replace the "gh-pages" branch.

I opted for replacing the gh-pages branch instead of creating a PR because this type of PR will inherently have conflicts.

MrBago commented 5 years ago

@mengxr I think this is ready for another review, I'm also working on a PR to update the _config.yml file as part of the release. I tried changing docs/_plugins/copy_api_dirs.rb to use version.sbt instead of the config file, but it seems that jekyll reads the config file too, so that only updated some of the version references but not all.

mengxr commented 5 years ago

The doc release script looks good. I'm merging this PR in. However, there might be an issue with doc build. I cannot find the public APIs in the generated Python API html page. Could you see if this happens on your laptop?

mengxr commented 5 years ago

This is the error I hit:

/mnt/sparkdl/python/docs/index.rst:15: WARNING: toctree contains reference to nonexisting document u' sparkdl.graph'
/mnt/sparkdl/python/docs/index.rst:15: WARNING: toctree contains reference to nonexisting document u' sparkdl.image'
/mnt/sparkdl/python/docs/index.rst:15: WARNING: toctree contains reference to nonexisting document u' sparkdl.transformers'
/mnt/sparkdl/python/docs/index.rst:15: WARNING: toctree contains reference to nonexisting document u' sparkdl.udf'
/mnt/sparkdl/python/docs/index.rst:15: WARNING: toctree contains reference to nonexisting document u' sparkdl.utils'
WARNING: autodoc: failed to import module u'sparkdl'; the following exception was raised:
No module named tensorframes
WARNING: autodoc: failed to import module u'graph.input' from module u'sparkdl'; the following exception was raised:
No module named tensorframes
WARNING: autodoc: failed to import module u'graph.utils' from module u'sparkdl'; the following exception was raised:
No module named tensorframes
WARNING: autodoc: failed to import module u'image.imageIO' from module u'sparkdl'; the following exception was raised:
No module named tensorframes
WARNING: autodoc: failed to import module u'transformers.utils' from module u'sparkdl'; the following exception was raised:
No module named tensorframes
WARNING: autodoc: failed to import module u'udf.keras_image_model' from module u'sparkdl'; the following exception was raised:
No module named tensorframes
WARNING: autodoc: failed to import module u'utils.keras_model' from module u'sparkdl'; the following exception was raised:
No module named tensorframes