Unable to select desired primary element

Description

I am consuming a JSON API feed from a third party with the following structure which is being used to try and import entries. However it appears the duplicated data keys are causing an issue with selecting the desired primary element to import.

{
    "success": true,
    "data": {
        "current_page": 1,
        "data": [
            {
                "id": 1,
                "name": "Event name",
                ...
            },
            {
                "id": 2,
                "name": "Event name",
                ...
            },
            ...
        ]
    }
    ...
}

I am looking to import the data array which as demonstrated below is not shown as an option.

I have set up a secondary feed to test the data structure and have renamed the data array to events which produces the following desired outcome:

{
    "success": true,
    "data": {
        "current_page": 1,
        "events": [
            {
                "id": 1,
                "name": "Event name",
                ...
            },
            {
                "id": 2,
                "name": "Event name",
                ...
            },
            ...
        ]
    }
    ...
}

I have also mapped the fields for this second test feed and successfully imported.

Unfortunately asking for the key to be renamed in the API is not an option. Data structures are identical but the duplicated data key seems to cause the issue. I am wondering if there's any workaround or explanation to why this might be as I couldn't find anything in the documentation.

Thanks.

Additional info

Craft version: 4.8.0
PHP version: 8.1
Database driver & version: MariaDB 10.4
Plugins & versions: Feed Me 5.4.0

Hi, thanks for getting in touch!

It’s happening because of this bit of code: https://github.com/craftcms/feed-me/blob/5.x/src/services/DataTypes.php#L269-L273 In your case the $tree contains the following keys: '/data', '/data/data', '/data/data/venue_address', '/data/links'. Line 269 is going to take the last part of each of those (parts are separated by a slash), and it’ll only add it to the array of available nodes if that value doesn’t already exist. In the case of '/data' and '/data/data', the last parts are the same - 'data' and so only the first instance is added to the array of available nodes to choose from.

If the keys can’t be renamed in the API, you can actually rename them yourself after the data has been grabbed from the feed. Assuming you have a JSON feed with the structure you mentioned earlier, you could write a module or a plugin and use the EVENT_AFTER_FETCH_FEED event to rename the second data key to, e.g. an event key. Here’s a bit of code that could get you started:

Event::on(DataTypes::class, DataTypes::EVENT_AFTER_FETCH_FEED, function(FeedDataEvent $event) {
    $response = JsonHelper::decode($event->response['data']);
    if (isset($response['data']['data'])) {
        $response['data']['events'] = $response['data']['data'];
        unset($response['data']['data']);
    }
    // This will set the feed's data
    $event->response = [
        'success' => true,
        'data' => JsonHelper::encode($response),
    ];
});

With the above, code in place, I'm able to select /data/entries as the primary element and import the data from the original /data/data node as expected.

More details about the events can be found here: https://docs.craftcms.com/feed-me/v4/developers/events.html#data-fetching-related-events.

I hope this helps! I'm going to close this now, but feel free to reach out if you run into any further issues.

craftcms / feed-me

Unable to select desired primary element #1414

Description

Additional info