Latest docker image

[jjkoehorst]

Just noticed and perhaps most importantly:

memote --version
memote, version 0.7.2

And this is a continuation of (as I ran into docker version issue)...

https://github.com/opencobra/memote/issues/260

I am working on the gitlab runner implementation and am working from the docker image to perform thorough testing.

docker run -it opencobra/memote:latest /bin/bash
pip install memote --upgrade

I had an old git repo which does not fit the current format so I created a new one using

memote new

<enter - enter - enter> all defaults

cd memote_model_repository

Following the documentation at:

https://memote.readthedocs.io/en/latest/getting_started.html#benchmark

Offline, local or private workflow:

memote run

critical: Working with a repository requires a storage location.

Using:

memote run --location location model.xml

Does work but the folder needs to exist. Perhaps we can automatically create the folder when it doesn't? (should I open a new ticket for this?)

When looking into using the history (keep in mind this is a new git repo using the default settings with no commit yet...)

memote report history

This will give a:

memote report history
Usage: memote report history [OPTIONS] LOCATION

Error: Missing argument "location".

which makes sense but is not clear in the documentation

Using memote report history Results/

in which the content is

root@a90da38c1e26:/opt/memote_model_repository# ls Results/
92be75a5e5db3f9c075d25745687350a74205c32.json

results in:

root@426583ce1a14:/opt/memote-demo# memote report history Results/
/usr/local/lib/python3.6/site-packages/memote/suite/reporting/config.py:57 UnsafeLoaderWarning: 
The default 'Loader' for 'load(stream)' without further arguments can be unsafe.
Use 'load(stream, Loader=ruamel.yaml.Loader)' explicitly if that is OK.
Alternatively include the following in your code:

  import warnings
  warnings.simplefilter('ignore', ruamel.yaml.error.UnsafeLoaderWarning)

In most other cases you should consider using 'safe_load(stream)'
Writing history report 'index.html'.
Loading result from 'Results/6108c6a27640402aaf33d498885e2d266b0735cb.json'.
Loading result from 'Results/73b7de8e4b0f13f40650c519df7184567bb9e533.json'.
Loading result from 'Results/b7fc3fdd57b3150f98aaf29715f65295e6428978.json'.
error: Could not load result 'b7fc3fdd57b3150f98aaf29715f65295e6428978'.
Loading result from 'Results/05f86dc0b6cc125d9e355ede64396d19d6378884.json'.
Loading result from 'Results/296f5416ea06323ae383a464a2d418f035bdc847.json'.
....
Loading result from 'Results/450597942d6924e5b51ebeef9fff46b53f4722b4.json'.
error: Could not load result '450597942d6924e5b51ebeef9fff46b53f4722b4'.
Loading result from 'Results/e5fed3fe755994c61f362c7fc3618983d4d3f610.json'.
error: Could not load result 'e5fed3fe755994c61f362c7fc3618983d4d3f610'.
Loading result from 'Results/39c6eba533fe4c1f49ef2f561faa193ba77cf71d.json'.
error: Could not load result '39c6eba533fe4c1f49ef2f561faa193ba77cf71d'.
Loading result from 'Results/eace9885c77f15e04dc128d43f26d1760cc064cc.json'.
error: Could not load result 'eace9885c77f15e04dc128d43f26d1760cc064cc'.

Traceback (most recent call last):
  File "/usr/local/bin/memote", line 11, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/memote/suite/cli/reports.py", line 150, in history
    api.history_report(repo, manager, filename, index=index, config=config)
  File "/usr/local/lib/python3.6/site-packages/memote/suite/api.py", line 127, in history_report
    file_h.write(report.render_html())
  File "/usr/local/lib/python3.6/site-packages/memote/suite/reporting/history.py", line 120, in render_html
    structure = self.collect_history()
  File "/usr/local/lib/python3.6/site-packages/memote/suite/reporting/history.py", line 91, in collect_history
    tests[test]["type"] = result.cases[test]["type"]
KeyError: 'type'

The basics works! but its still a work in progress

The gitlab repository used: https://gitlab.com/wurssb/memote-demo/tree/master

The runner file used: https://gitlab.com/wurssb/memote-demo/blob/master/.gitlab-ci.yml

The resulting GitHub-pages: https://gitlab.com/wurssb/memote-demo/tree/gh-pages/ Gitlab is not doing anything with them but I can make it so that it will make use of the gitlab pages if requested?

The entire git run: https://gitlab.com/wurssb/memote-demo/-/jobs/75803125

The nice thing is that it can make use of the gitlab computational resources. Not sure if travis is hosted by GitHub or you need to set this up yourself?

Will update soon with new information but the documentation looks a bit off..

[Midnighter]

Hi @jjkoehorst, thank you for the thorough report. I'm still finishing up #441 and #132 which will update memote new and correct the memote history + report. The documentation also needs updating, this is tracked in #379 and #417.

I had noticed the docker image not being updated properly. This is a problem with the new PyPI that takes much longer until a new deployed version is available and thus breaks the build of the docker image. Can you open an issue for that, please?

[jjkoehorst]

Just saw the email... :) and I would like to continue with the integration of the gitlab base. Any news on the docker image and latest memote?

[Midnighter]

The docker image is now being built by a (daily) cron job on Travis. At the moment the image is only built when a new memote version is out but I'm thinking of changing that to just rebuild daily such that dependencies get updated once in a while.

The new workflows are documented if they have changed at all.

What else is needed for you to proceed with Gitlab?

[jjkoehorst]

Great!... I see you made it such that a normal user is running in the docker image? I might need to install some additional tools but since it has no sudo rights I am running against a wall..

https://gitlab.com/jjkoehorst/memote-demo2/blob/master/.gitlab-ci.yml

'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'

[Midnighter]

Would you mind making a PR against https://github.com/opencobra/memote-docker/blob/master/Dockerfile to adjust the Dockerfile as needed?

I think for now I prefer carrying extra dependencies in the memote image as long as they can be justified by enabling additional workflows. I would like to avoid running memote as root inside the container.

[jjkoehorst]

Basics seems to work now:

https://gitlab.com/jjkoehorst/memote-demo3

I am using the following configuration: https://gitlab.com/jjkoehorst/memote-demo3/blob/master/.gitlab-ci.yml (can create a pull request for that when we all agree on the commands used (maybe there are things that I have missed or are redundant)

Using the following variables $GITREPO and $SSH_PRIVATE_KEY

I will write a more extensive documentation on how to generate a private and public key. Where to add them on gitlab. How to change the git url that memote makes to a new empty repo on gitlab. Set the variables needed for the runner.

I am currently looking into the gh-pages and how to use this in gitlab as it is possible.

https://jjkoehorst.gitlab.io/memote-demo3/

But it does not seem to show everything? I only used the index.html file.

and this is the runner for the gh-pages:

https://gitlab.com/jjkoehorst/memote-demo3/blob/gh-pages/.gitlab-ci.yml

[Midnighter]

What's the best way for me to comment on that gitlab runner configuration?

[jjkoehorst]

Very good question

Not sure actually. We can make a google document? I don’t see line comments options yet maybe via a pull request?

On 14 Aug 2018, at 11:13, Moritz E. Beber notifications@github.com wrote:

What's the best way for me to comment on that gitlab runner configuration?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/opencobra/memote/issues/453#issuecomment-412807654, or mute the thread https://github.com/notifications/unsubscribe-auth/AEQ7kwEzUJeMbMICxh8LIcKUm-8FVWHmks5uQpTHgaJpZM4Us3d0.

[Midnighter]

Okay, I'll make a pull request. Prefer that to a Google doc.

[jjkoehorst]

You can only comment on diffs not on full files so I just made a full file commit:

https://gitlab.com/jjkoehorst/memote-demo3/commit/660bc41d556146ec9650390350760afbfb7502d3