search

craftcms / feed-me

Craft CMS plugin for importing entry data from XML, RSS or ATOM feeds—routine task or on-demand.
Other
288 stars 139 forks source link

Implement naive feed configuration storage #1399

Open lthurston opened 8 months ago

lthurston commented 8 months ago

Description

This is the beginning of an idea to read / write feed configuration from / to the filesystem. This allows feeds to be tracked in the filesystem, like any other configuration or code. Developers will be able to create the feeds locally, write the feed to the filesystem using ./craft feed-me/feed-config-storage/write, commit that file, and, as a deploy step, read the configuration from disk with ./craft feed-me/feed-config-storage/read.

This hasn't been fully tested. I assume there are glaring bugs. Is this a feature that might be considered valuable enough to be incorporated into the plugin, or are we unique in this need? We can, but prefer not to, maintain our own fork, but I'd rather share this work with others.

I'm curious to hear your thoughts and whether you can offer any course corrections or other feedback. Thanks!

Note: you will need to mkdir config/feed-me for this to work at the moment.

tricki commented 7 months ago

FYI I manually extracted some feeds but ran into some problems when deploying to different servers with different databases. The main problem was the fact that the configuration contains auto-generated IDs (e.g. siteId), which might be different on different deployments. In my current solution I still have to configure those manually (site, section type and entry type, if I remember correctly).

lthurston commented 3 months ago

Hi @tricki, in 80f6667 I switched from ids to uids for the very reason you mention, which I think should solve your problem. I'm not doing anything with siteId in my changes, but we have implemented my forked version of this repo in a project, and it works.

It's important to make sure the ./craft feed-me/feed-config-storage/read CLI command is executed as a deploy step, and when your local dev env starts up too.

AugustMiller commented 2 weeks ago

This is very cool! Thanks for sharing with us.

I suspect that the greatest utility here is portable feed definitions, which is absolutely worth considering—but I'm not sure how to square the current implementation against Craft’s built-in Project Config system, or the community's desire to have feeds managed alongside schema/source code instead of as content. In keeping with Craft’s workflow conventions, the former would require the involvement of a developer any time a feed changed… and while I can see a lot of value in that, it may bbe a non-starter for a lot of clients who need to run and tweak imports.

Happy to leave open for discussion + feedback!