AFC-1406 - API Exchange data between CRM and DAFA (formerly Fenix)

[FredrikArvas]

Userstory: As a user, I want my Jobseekers and their projects to be syncronized between CRM and FENIX

We are talking about different solutions of updating data between our CRM-system and the Fenix-system (two instances of Odoo CE 12). In this case we are talking about syncronizing changes on the jobseeker and there projects (including tasks and subtasks.)

First iteration: 1 Create a Project (linked to a Jobseeker) from Fenix to CRM 2 Update changes in a Project (linked to a Jobseeker) from Fenix to CRM Modules involved: Project Contacts (not the sales module)

Requirements odoo-af/af_sale_orders

Fields (simple listing) project.id project.name project.description project.state project.partner_id (might need to be created) project.user_id (might need to be created) project.subtask_project_id project_task.id project_task.name project_task.state project_task.date_start project_task.date_start project_task.description

Second iteration: 2 Create Jobseeker in Fenix, when a Sales Order is recieved relating to a new Jobseeker 3 Update a Jobseeker in Fenix, when changes are made in CRM 4 Update a Jobseeker in CRM, when changes are made in Fenix

Module name: (added by Nils or Fredrik) odoo-api/api_project_odoo_xmlrpc

Branch name: https://github.com/vertelab/odoo-api/tree/Dev-12-Fenix-Sprint-02

Module type: Application/Property/Resource/Glue module [see https://github.com/vertelab/odoo-af/issues/178 for more info]

Integration:

Test-steps: 1 Create a new Project in the source system called "Project 1 Server" 2 Create a new Task linked to the "Project 1 Server" in the source system called "Project 1 Task" 3 Check for the synchronized project on the destination system.

Test-data:

Questions

1 What happens if a child-record (on the sending-side) is copied to the receiving-side without it's parent-record? Suggestion: It is excluded and reported in the log. It will be synchronized in the next run. = If child record copied without it's parent record then can't see relation ship between child and parent.

2 Is there a way to filter the sync for changes between YYYY-MM-DD to YYYY-MM-DD? To limit the sync-time to records that have been changed not sync all records every time.

3 what happens with the "last changed by"-field? I understand the date-field. I wandered about the field where the "last changed by user-field". Scenario: We have one CRM-server (receiving side) and one Fenix-server (sending side). We have different set of users on each server. Userstory: As a server admin, I (think I that I) want Tims user id to be transferred to the receiving side (fields create_uid and write_uid) when Tim creates a record on the Fenix-server.

4 Userstory: As a admin, I want to specify which records should be synced to which database. Scenario: We have one CRM-database and 200 Fenix-databases. Only a few jobseekers should be synced to each database.

Description: This module adds xmlrpc-based sync between to servers' databases' models.

• Configre the servers: https://www.loom.com/share/eea2043574de49659fe89e9b569eac36
• Configure objects to sync: https://www.loom.com/share/9798ae8150c5469a9a11495dc0e77345
• Run syncronisation dowload https://www.loom.com/share/d8c1f90df6cd47469d8365daf7aa601c
• Run syncronisation upload https://www.loom.com/share/1742741777734209b150bb131683fd82

Estimate: (added by developers only) Estimated Hours: Estimated Date: Actual Hours: 14, Nov 2020 - Issue #361: AFC-1406 Working on to analysis for exchange data. - 30 mins 18, Nov 2020 - Issue #361: AFC-1406 Analyzing on Sync module. - 30 mins 21, Nov 2020 - Issue #361: AFC-1406 Working on to module for exchange data. - 3 hours and 30 mins 23 Nov 2020 - Issue #361: AFC-1406 Working on to module for exchange data. - 3 hours and 40 mins 26 Nov 2020 - Issue #361: Checking Comments. - 20 mins 02 Nov 2020 - Issue #361: AFC-1406 Working on modification of exchange data module. - 1 Hour and 20 mins 03 Nov 2020 - Issue #361: AFC-1406 Working on modification of exchange data module. - 20 mins

[FredrikArvas]

@AnilKesariya @rupareliyahemangi145 Have you worked with these integrations before? We think xmlrpc would be the best option. Which information do you need us to specify in order for you to build a "server"- and "client"-module for communication?

[rupareliyahemangi145]

Hello @FredrikArvas

Yes we have worked on integration before. Yes XMLRPC is best option. We need following information:

1. URL of both server Fenix and CRM.
2. Admin user's username and password.

[FredrikArvas]

Ok, Please - help me understand the process. (because I have not done this before). Is this done with two modules (one for each side)?

Do we need to servers to test it, or can it be tested with two databases on the same server?

How much time do you estimate the work for iteration 1 and iteration 2?

Thanks

[rupareliyahemangi145]

Hello @FredrikArvas

It can be tested with two databases on same server. I will create one module that will ask some information like url, db, user name, password of server or database from which you want to fetch data. We will create Scheduled actions to fetch data from one database or server to create in current database where schedule action is running.

Time estimation is depends on how much records, how much models and how much fields.

[FredrikArvas]

Ok. Could we start with: the the project you have worked with?

First iteration: 1 Create a Project (linked to a Jobseeker) from Fenix to CRM 2 Update changes in a Project (linked to a Jobseeker) from Fenix to CRM

For fields - see updates in the description above. We need to remember complexity because we need to manage the sync between one CRM-database and multiple FENIX-database.

[rupareliyahemangi145]

Hello @FredrikArvas

Thanks for information of fields. I want to ask one question, In first iteration you want to create project and its tasks in CRM. So is there possibility that project in CRM is already created which are in Fenix and in CRM need to update if exits? or we just want to create new projects and its details in CRM from Fenix don't need to check exits or not?

Instead of creating module we can also create with odoo base import and export functionality. if need to create only new. if we can update which are created from import if need. It would be easy compare to create new module for schedule action and to create. I can't direct use other project module which I've made. I can use it just for reference.

[FredrikArvas]

Hi again @rupareliyahemangi145

The projects we are talking about will only be created in one of the 200 databases on the Fenix-server. We then want them to be replicated to the single database in the CRM-server, with read-only access. When any of the projects or their tasks are updated on the Fenix databases, we want the changes to be replicated to the CRM-database.

Does this answer help?

[rupareliyahemangi145]

Hello @FredrikArvas Yes this answer help. But can we schedule meeting to discuss this issue?

[FredrikArvas]

Hi @AnilKesariya Could you please give us an update about this issue when you have time. As agreed on the phone, we would like a "settings"-page where we can enter the source- and the target-system. Something similar to this module's config page: https://github.com/vertelab/odoo-af/tree/Dev-12.0/af_ipf

Thanks

[rupareliyahemangi145]

Hello @FredrikArvas

We are working on this issue. Yes there will be setting page where you can configure systems, objects.

We will send you video of functionality as soon as possible.

Thanks.

[rupareliyahemangi145]

Hello @FredrikArvas

Here is videos to show what we have done for this issue.

1. configre the servers: https://www.loom.com/share/eea2043574de49659fe89e9b569eac36
2. configure objects to sync: https://www.loom.com/share/9798ae8150c5469a9a11495dc0e77345
3. run syncronisation dowload https://www.loom.com/share/d8c1f90df6cd47469d8365daf7aa601c
4. 3. run syncronisation upload https://www.loom.com/share/1742741777734209b150bb131683fd82

Some points are remaining like need to create relation fields(Many2one and One2many) if does not exits in local db(Where getting data) Plus now need to sync Project and Task one by one instead of that this should be at a time. We can sync at a time both but if syncing at a time can't set project reference in task.

So, these points plus some more testing are remaining.

Please check it and let me know if anything.

Thanks.

Regards, Hemangi.

[FredrikArvas]

Thanks @rupareliyahemangi145 - I will look and come back soon.

[rupareliyahemangi145]

Thanks @FredrikArvas

One more thing I forgot to mention is I've created only one module which need to install in both database. Need to configure server, objects and Synchronize where we want data.

[FredrikArvas]

@rupareliyahemangi145 This works fine, since it is the "config" that decides if it is a server or a client.

[rupareliyahemangi145]

Ok @FredrikArvas

[FredrikArvas]

Thanks, Please update the amount of time for this issue.

[FredrikArvas]

Hi @rupareliyahemangi145 1 What happens if a child-record (on the sending-side) is copied to the receiving-side without it's parent-record? Suggestion: It is excluded and reported in the log. It will be synchronized in the next run.

2 Is there a way to filter the sync for changes between YYYY-MM-DD to YYYY-MM-DD? To limit the sync-time to records that have been changed not sync all records every time.

3 what happens with the "last changed by"-field?

[rupareliyahemangi145]

Hello @FredrikArvas

1. If child record copied without it's parent record then can't see relation ship between child and parent.

2. I need to check this.

3. You mean Latest Synchronization date? It stores date of last sync.

[rupareliyahemangi145]

Spent time details:

14, Nov 2020 - Issue #361: AFC-1406 Working on to analysis for exchange data. - 30 mins 18, Nov 2020 - Issue #361: AFC-1406 Analyzing on Sync module. - 30 mins 21, Nov 2020 - Issue #361: AFC-1406 Working on to module for exchange data. - 3 hours and 30 mins 23 Nov 2020 - Issue #361: AFC-1406 Working on to module for exchange data. - 3 hours and 40 mins 26 Nov 2020 - Issue #361: Checking Comments. - 20 mins

[FredrikArvas]

Hi again @rupareliyahemangi145

Question: 3 what happens with the "last changed by"-field? I understand the date-field. I wandered about the field where the "last changed by user-field". Scenario: We have one CRM-server (receiving side) and one Fenix-server (sending side). We have different set of users on each server. Userstory: As a server admin, I (think I that I) want Tims user id to be transferred to the receiving side (fields create_uid and write_uid) when Tim creates a record on the Fenix-server.

4 Userstory: As a admin, I want to specify which records should be synced to which database. Scenario: We have one CRM-database and 200 Fenix-databases. Only a few jobseekers should be synced to each database.

[FredrikArvas]

@AnilKesariya and @rupareliyahemangi145 Just wanted to say I that I am really pleased about the work you do with us! thanks a lot.

[rupareliyahemangi145]

Hello @FredrikArvas

Userstory: As a server admin, I (think I that I) want Tims user id to be transferred to the receiving side (fields create_uid and write_uid) when Tim creates a record on the Fenix-server.

• This worked if User created in Receiving side.If not created user then need to create it which is remaining.

Videos that I sent you it just demo we can change functionalities.

[rupareliyahemangi145]

@AnilKesariya and @rupareliyahemangi145 Just wanted to say I that I am really pleased about the work you do with us! thanks a lot.

Thanks Fredrik.

[FredrikArvas]

Hi @rupareliyahemangi145 Is it possible for you to do a first PR of this module tomorrow befor your lunch? It would really help us on our side. It does not need to be ready, but we would like to test it in our environment to understand how it works and how we should set it up. Thanks

[rupareliyahemangi145]

Yes Sure @FredrikArvas I will create PR for this issue.

[rupareliyahemangi145]

Hello @FredrikArvas

Created PR https://github.com/vertelab/odoo-api/pull/1. Created only one module api_odoo_xmlrpc. Need to install module in both database where receiving and sending. But configuration need only in receiving db as shown in videos.

Please check it and let me know if anything.

Thanks.

Regards, Hemangi.

[FredrikArvas]

@rupareliyahemangi145 Thanks, It will be so fun to test this! :-)

[rupareliyahemangi145]

Ok @FredrikArvas :)

[FredrikArvas]

Hi @rupareliyahemangi145 This is getting REALLY GOOD!

Some improvements/adjustments:

1 Move the links from the top-menu to a Section in the General settings-page Add debug-for Sync-config-menu-item

2 Please add config-example-data-file (in state "inactive") for for all settings to the /data-folder (not the demo-folder)

3 Add Active/Inactive to all Sync-tree-views

4 Low prio: Please add Sync-schedule on object-level

5 Optional: Add dropdown in field-exclusion-list. (I see that the fields are visible thorught the icon right to the Model)

[rupareliyahemangi145]

Hello @FredrikArvas

I've create PR https://github.com/vertelab/odoo-api/pull/5 for point 1, 3 and 5. Here is video: https://www.loom.com/share/cc7b55165a5d49d685a8b35161f754de

Please check it and let me know if anything.

Point 2, is not possible as Server is required field so we need to create server also from data folder(Demo Data) and can't add data from data folder we need to create it from hook(When installing module it adds data but can't add new from code with module upgrade) plus need to give modules in depends for which want to add data.

Point 4, Sync-Schedule means which menu's data?

Thanks.

Regards, Hemangi.

[FredrikArvas]

Hi @rupareliyahemangi145 Thanks for update and answers.

• Point 4, Sync-Schedule means which menu's data? Here I am trying to illustrate that I would like to configure a schedule for the syncronization for each object on the "Sync-item-list".

Background: We have two types of sync-situations: "On Event" and "On Schedule". We have two sync scopes: "Selected record(s)" and "All records".

On this item, I wanted to add the "On Schedule"-setting on a "syncronized object".

Example: Task-categories change slowly, and need only to be synced once every night. Phone-numbers change frequently, they need to be synced "every 3 hours"

[FredrikArvas]

@rupareliyahemangi145 Is there a "action" on the syncronized action that we can "call" on when a record is changed? Example: As a user, I update the phone-number of a contact. When that record is saved, I want the sync to run for that record only.

[rupareliyahemangi145]

Hello @FredrikArvas

I need to check this. I will check it and get back to you soon.

[rupareliyahemangi145]

Hi @rupareliyahemangi145 Hello @FredrikArvas We have two sync scopes: "Selected record(s)" and "All records".

• For Selected record you can add Domain in Synchronized objects. FA: Yes please.

On this item, I wanted to add the "On Schedule"-setting on a "syncronized object". Example: Task-categories change slowly, and need only to be synced once every night. Phone-numbers change frequently, they need to be synced "every 3 hours"

• We can add menu where you can configure server, server objects and synced time(Hour) and I will create Scheduled Action which run every 1 hour and it will sync records which matches with current hour. FA: Yes please.

Is there a "action" on the synchronized action that we can "call" on when a record is changed? Example: As a user, I update the phone-number of a contact. When that record is saved, I want the sync to run for that record only.

• For this I didn't find any solution I need to investigate more. FA: OK, please do! Thanks.

[arturchilangwa-vertel]

Is now merged https://github.com/vertelab/odoo-api/pull/5