Medium Sync

[fabioscarsi]

Hi guys, thanks for your plugin. I've been using it for a while. Considering the release of Obsidian Web Clipper, this interesting and well-working plugin risks becoming soon obsolete. @arumie and @dzatrifork, could you imagine a development that transforms Obsidian Medium Importer into an app that automatically syncs an Obsidian folder of articles with the whole set of articles published by a Medium writer? Is it possible? This way, combined with https://github.com/eebmagic/post-medium-draft, we would have a killer suite in Obsidian to work with Medium ...

[arumie]

Hi. Glad to hear that you like the plugin 😊

The Unofficial Medium API definitely seems to be support fetching articles made by a specific user.

https://docs.mediumapi.com/#tag/User/paths/~1user~1%7Buser_id%7D/get

Was also considering adding the possibility of sync a list made by a user.

This would of course be limited by the amount of free API calls available each month.

Might take a look at adding these features if I find the time 🙂

[fabioscarsi]

Hi @arumie. Thanks a lot for your positive answer. I would really appreciate your development(s), so I hope you can find the time. If I were a coder, I would help you for sure. Just a consideration on quantitative limits: also systematic writers write up to one post daily on Medium, but no more. That's 30 posts a month. The free API calls are 150/month. Probably, it could be enough. Super heavy users could consider paid options ... One more thing: I have a technical doubt. Medium has a paywall. Does it limit fetching by the Unofficial Medium API?

[arumie]

No better reason to get it done than someone actually wanting it, so I also hope that I can find the time. Might even take a look at it this or next weekend. :)

I'm of the same opinion with regards to the free API calls. Heavy users definitely should consider supporting the developer with the paid option anyway.

At the moment it doesn't seem like the paywall is a limit. I think the guy who develops the API is probably the best to answer that question (https://links.nishu.io/).

[fabioscarsi]

Thanks, @arumie. Regarding the paywall question, reading the FAQs (https://mediumapi.com/frequently-asked-questions.html), particularly the last one, it seems you are right, and the paywall is not a problem. Anyway, I wrote the developer and will confirm it with you as soon as he answers.

[arumie]

I've added a new feature with the new version 0.3.0: "Add Medium Author" and "Sync Medium Author". I've also added the possibility to automatically add the articles to a specific folder (Folder is "Medium" by default). The author feature will add a folder for the specific author and add all their articles to that folder. Sync with add any articles that hasn't already been added to the folder.

Let me know what you think :)

[fabioscarsi]

Wow! @arumie, I'm impressed by your speed. Hat off. I've tried the new version on my system: Obsidian V1.7.4 on MacOS Sequoia 15.0.1.

1. Importing a single post from Medium works almost perfectly (I had just a problem with a failure to import with a single specific post).
2. For single post clipping/fetching, your plugin is an alternative to Obsidian Web Clipper. Comparing the two: a. The web clipper now is more convenient because it does not impose some steps necessary with your plugin; you have just to push a button in the browser; b. The results are almost identical, but your plugin is better with formatting and pictures. At the same time, the web clipper has more flexibility in renaming the file and setting properties. Quite soon, AI will be officially available if desired. (I uploaded two screenshots of the same note imported with your plugin and the clipper as an example, as well as the JSON template for the clipper that I set to clip Medium articles if you'd like to take a look). my-medium-articles-clipper.json
3. Introducing a syncing functionality in your plugin is a breakthrough. But right now, I can't manage to complete the workflow: I set everything, and I launch the command "Sync Medium author," the plugin tells me how many articles there are to sync ("[Medium Importer] Found 30 articles by Fabio Scarsi"), but nothing happens, articles are not downloaded (consider the plugin works without problems for single imports with "Import article from Medium").
4. All these things considered, my humble opinion is that the way you chose is correct. The plugin could evolve in the direction you have found with some progressive improvements and rationalizations (just my opinion, obviously): 5.1. Setting authors for fetching should be done in the settings panel, where it would be helpful also to be able to define specific folders for each author by hand; 5.2 the command "Sync Medium author" is handy for sure, but in my opinion, it would also be more helpful to set periodic syncs in your plugin's settings (e.g., daily ...). The two options could coexist; 5.3 In the medium-long period, you could also add some functionalities, such as prepending (e.g., the publication date to the filename for correct temporal ordering) or some way to extract and manage properties (you can see what I mean in the screenshots and the alleged JSON file).

Anyway, thanks a lot for continuously developing your plugin. (And do you have any suggestions for solving my fetching problem to start some debugging?)

[arumie]

Hi @fabioscarsi

Thank you for you feedback. I've just tried it out myself and I think that I see the problem.

https://medium.com/@fabioscarsi/we-shall-not-cease-from-exploration-and-the-end-of-all-our-exploring-will-be-to-arrive-where-we-1c63dbbd57d1

This article doesn't have a title, which I thought always would be there (I'm using some text splitting to try and find it from the markdown). I think I have an idea on how to fix it. So I'll get back to you with a fix :)

Also a great idea with the properties. I might just add them while I'm at it. Might cost an extra API call, but I think it's worth it. :-)

With regards to having the authors in the settings. The Obsidian guys are pretty strict about what is allowed to be in the settings. But I do see what you mean. Maybe I can figure something out at a later date which is a bit easier to manage.

[fabioscarsi]

Thanks @arumie I hope not to raise the bar excessively and pretend too much. You are definitely busy with something else. Just consider my feedback as some brainstorming, but I am glad to try the various solutions you are experimenting with.

[arumie]

I think it'll work if you update the plugin now. It did when I tested it with your username on Medium Added the authors to the settings as well, including the possibility to set each author on "autosync", where it'll sync on startup :)

I'll write the one with the directory for each author on the backlog for now, but might get back to that at some point.

I appreciate the feedback and help. It's difficult to know whether the plugin actually works, if there isn't anyone who gives feedback, so thank you.

Both features haven't taken much of my time, so no problem on that front. Just glad that someone actually uses something I made for fun. :-)

[fabioscarsi]

Great Job @arumie. Thanks. Now, it works and fetches all the articles smoothly. Without any urgency but just as a memo, consider these details that still don't work, at least In my setting:

1. The selector in plugin settings doesn't keep the green status to sync the author on startup;
2. in properties, fetching my articles, the author field is not populated with "Fabio Scarsi", my author name, but I see a number (20a671a3756b);

3. Before inserting properties in the workflow, your plugin also downloaded the pictures in my medium posts. Now, it doesn't, neither in the automatic author fetching nor downloading a single post through its link. I uploaded a screenshot as an example.

   

[arumie]

Hi @fabioscarsi

Sadly the API doesn't send the author name with the article info, but the author ID. I didn't want to use an extra API call in the individual import, just for the name.

With regards to the images, it seems to be the markdown coming from the API when the images has captions with links in them. It results in the following markdown: ![Image by [Fellow Consulting AG](https://www.fellow-consulting.de/)](https://miro.medium.com/1*OAHUmDf7SkFFeYF6Bsa5EQ.png)

Which isn't valid. I don't really want to mess too much with the markdown after getting it from the API, as this might break other stuff. But its manually fixed by removing the link from the image link in the markdown.

Settings should be fixed now :)

[fabioscarsi]

Hi @arumie. Yes, the switch in settings is ok. Thanks. Great job! Now, Obsidian is a very sophisticated and functional editor and backup system for Medium writers. The definitive version of the articles is available in the vault for all the plugins ecosystem (e.g., Smart Connections and other AIs). It is a great integrated tool, also thanks to your plugin.

(I have just one more doubt. Doesn't the plugin already know the author when fetching? I haven't read the code, but can't the plugin overwrite that specific field, which is fixed for default in that API call? Anyway, keep this idea for some future review of the code.)