libero / walking-skeleton

A walking skeleton of the scholarly publishing system
MIT License
7 stars 2 forks source link

ELPP-3621 Downstream service sample: emit one event in response to article #25

Closed giorgiosironi closed 6 years ago

giorgiosironi commented 6 years ago

For #29

See README for how much is implemented in this PR and what will follow.

Sample log:

downstream-sample_1                | Receiving: {'eventId': '7d5de096-6338-11e8-83a9-0242ac1a000c', 'happenedAt': '2018-05-29T12:04:50+00:00', 'aggregate': {'service': 'article-store', 'name': 'article-version', 'identifier': '0065'}, 'type': 'article.version.post.started', 'data': {'article_version': 1}}
downstream-sample_1                | Publishing: {'runId': None, 'type': 'downstream.crossref.started', 'message': 'We are happy to receive this paper and have put it into a queue', 'eventId': '72d65f3a-6339-11e8-b721-0242ac1a000a', 'happenedAt': '2018-05-29T12:11:42+00:00'}
downstream-sample_1                | Receiving: {'eventId': '7d6ef94e-6338-11e8-83a9-0242ac1a000c', 'happenedAt': '2018-05-29T12:04:50+00:00', 'aggregate': {'service': 'article-store', 'name': 'article-version', 'identifier': '0065'}, 'type': 'article.version.post.completed', 'data': {'article_version': 1}}
downstream-sample_1                | Publishing: {'runId': None, 'type': 'downstream.crossref.started', 'message': 'We are happy to receive this paper and have put it into a queue', 'eventId': '72d8925a-6339-11e8-b721-0242ac1a000a', 'happenedAt': '2018-05-29T12:11:42+00:00'}
seanwiseman commented 6 years ago

Seem __pycache__ directory is being commited

giorgiosironi commented 6 years ago

I'll be going on implementing feedback on Wednesday, assuming the general framework of #28 is confirmed.

giorgiosironi commented 6 years ago

Integrated with article store from master (only issue the missing runId), will also check the dashboard after it has changed to the new format.

giorgiosironi commented 6 years ago

Merged master, but the article store doesn't start (no logs are written) as requests to the container's 8000 port are refused despite the python manage.py runserver process being running.

giorgiosironi commented 6 years ago

It starts when running with docker-compose up article-store, but docker-compose up -V reproduces the problem for me.

giorgiosironi commented 6 years ago

Non-deterministic problem.

giorgiosironi commented 6 years ago

Some manual tinkering lead to:

Run 4df2280c-67f8-11e8-9208-0242ac1a0010
    event.article.version.post.started
    Jun 4, 2018, 1:07:59 PM
    event.downstream.crossref.started
    We are happy to receive this paper and have put it into a queue
    Jun 4, 2018, 1:07:59 PM
    event.article.version.post.completed
    Jun 4, 2018, 1:07:59 PM
    event.downstream.crossref.started
    We are happy to receive this paper and have put it into a queue
    Jun 4, 2018, 1:07:59 PM
giorgiosironi commented 6 years ago

Had to manually bind the queue to the second exchange however: https://github.com/php-amqplib/RabbitMqBundle/issues/367

giorgiosironi commented 6 years ago

Thought about adding depends_on: - downstream-sample for the dashboard-events, but it doesn't have that for the articles exchange and it's just a startup order (or build time) dependency, so wouldn't add any value.

thewilkybarkid commented 6 years ago

Can see it working, but seems to respond to any article event? I see two downstream.crossref.started events: one after article.version.post.started and one after article.version.post.completed. Would only expect to see the latter.

seanwiseman commented 6 years ago

Successfully receiving the downstream events locally.

Are we going to have some sort of filtering on the received events here? e.g. only act on type article.version.*.completed or something along those lines? I know its not strictly important for demo purposes but with its current implementation, once you add in the work flow steps it will be firing off downstream.crossref.started about 10+ times when realistically you would only want to trigger downstream once the initial article 'processing' has completed.

giorgiosironi commented 6 years ago

I'll put a filter on article.version.post.completed that seems the correct integration point.

giorgiosironi commented 6 years ago

Took a bit to understand the flow of routing keys but :+1: feel free to merge when conveniente.