Multimedia support phase 0

[nonumpa]

Spec: https://g0v.hackmd.io/aJqHn8f5QGuBDLSMH_EinA

Schema

https://github.com/cofacts/rumors-db/pull/56

• Article add attachmentUrl, attachmentHash, articleType(ArticleTypeEnum)

Method

CreateMediaArticle

(Only supports image currently.)

• Input an URL as mediaUrl and an articleType to create an article.
  • It will create a new article if the hash of the image is not found in any articles.
  • If found, return first matched article id.

CreateArticle

• Add articleType: TEXT, attachmentHash: '', attachmentUrl: ''

ListArticles

• Default returns TEXT type articles
• Add mediaUrl filter to get the article which attachmentHash is similar to mediaUrl's
• Add articleTypes filter to get certain types of articles

Setup

Cloud storage

1. Create a bucket and paste the name to GCS_BUCKET_NAME, you can create a folder GCS_IMAGE_FOLDER to save image files or leave it null to root.
2. Make data public https://cloud.google.com/storage/docs/access-control/making-data-public?hl=zh-tw#console
3. Get the cloud storage JSON key and paste it to GCS_CREDENTIALS. https://cloud.google.com/docs/authentication/getting-started#cloud-console

Migration

Add a articleType="TEXT" to those articles which don't have a articleType.

curl -XPOST "{db_url}/articles_{vesrion}/_update_by_query" -H 'Content-Type: application/json' -d'
{
   "script": {
    "source": "ctx._source[\"articleType\"]=\"TEXT\";",
    "lang": "painless"
  },
  "query": {
    "bool": {
          "must_not": {
              "exists": {
                  "field": "articleType"
              }
          }
      }
  }
}'

CreateMediaArticle curl :

curl 'http://localhost:5000/graphql?userId=dev' -H 'Accept-Encoding: gzip, deflate, br' -H 'Content-Type: application/json' -H 'Accept: application/json' -H 'Connection: keep-alive' -H 'DNT: 1' -H 'Origin: http://localhost:5000' -H 'x-app-secret: foo' --data-binary '{"query":"mutation(\n  $mediaUrl: String!\n  $articleType: ArticleTypeEnum!\n  $reference: ArticleReferenceInput!\n) {\n  CreateMediaArticle(\n    mediaUrl: $mediaUrl\n    articleType: $articleType\n    reference: $reference\n    reason: \"\"\n  ) {\n    id\n  }\n}\n","variables":{"mediaUrl":"http://localhost:8007/url.jpg","articleType":"IMAGE","reference":{"type":"LINE"}}}' --compressed

graphql:

mutation(
  $mediaUrl: String!
  $articleType: ArticleTypeEnum!
  $reference: ArticleReferenceInput!
) {
  CreateMediaArticle(
    mediaUrl: $mediaUrl
    articleType: $articleType
    reference: $reference
    reason: ""
  ) {
    id
  }
}

{
  "mediaUrl": "url.jpg",
  "articleType": "IMAGE",
  "reference":  { "type": "LINE" }
}

[coveralls]

Coverage increased (+0.4%) to 87.533% when pulling dab9586ddc178aaa919ca566041d3a57e09fce56 on features/image-collection into a34e025ad230b50367306758d145453be9a9bd51 on master.

[MrOrz]

Migration report on staging

Update article schema (in rumors-db)

$ npm run reload -- articles

> rumors-db@1.0.2 reload
> babel-node db/reloadSchema.js "articles"

Source:  articles_v1_1_0
Target:  articles_v1_2_0
Reindexed from articles_v1_1_0 to articles_v1_2_0 in 39 seconds.
Setup articles_v1_2_0 -> articles alias and remove articles_v1_1_0.

Fill in article type: call

curl -XPOST "{db_url}/articles_{vesrion}/_update_by_query" -H 'Content-Type: application/json' -d'
{
   "script": {
    "source": "ctx._source[\"articleType\"]=\"TEXT\";",
    "lang": "painless"
  },
  "query": {
    "bool": {
          "must_not": {
              "exists": {
                  "field": "articleType"
              }
          }
      }
  }
}'

took 53 second on 67k articles

{"took":53524,"timed_out":false,"total":67680,"updated":67680,"deleted":0,"batches":68,"version_conflicts":0,"noops":0,"retries":{"bulk":0,"search":0},"throttled_millis":0,"requests_per_second":-1.0,"throttled_until_millis":0,"failures":[]}