Add patchwork interface

[rsarky]

This PR adds supoort to pull patches from patchwork. There are 2 ways PaStA can pull patches:

• A raw mailbox containing patches from patchwork. (can be used for initial import)
• Through Patchwork's API. In this case Pasta will only import patches that it doesnt have already.

[rralf]

Thanks! I'll have a look at it tomorrow.

[rralf]

One thing: Could you take the adjustments of Config.py and load/write_index to a separate commits?

[rsarky]

One thing: Could you take the adjustments of Config.py and load/write_index to a separate commits?

Sure

[rsarky]

Updated

[rsarky]

Hi,

before I continue with my review: please remove unnecessary if len() checks, and take care of the indentation. In all cases, you can remove the else statement. While it's surely not a good idea, there's no need to disallow raw mboxes in combination with patchwork. It's not required for our use-case, but there's no reason to forbid it.

In all the cases, the else and the conditions based on len were introduced to disallow PaStA using Patchwork and public or raw mailboxes simultaneously. If we do want them to function together, I guess I can remove those checks

[rralf]

Hi, before I continue with my review: please remove unnecessary if len() checks, and take care of the indentation. In all cases, you can remove the else statement. While it's surely not a good idea, there's no need to disallow raw mboxes in combination with patchwork. It's not required for our use-case, but there's no reason to forbid it.

In all the cases, the else and the conditions based on len were introduced to disallow PaStA using Patchwork and public or raw mailboxes simultaneously. If we do want them to function together, I guess I can remove those checks

Yep, let's remove them.

[rsarky]

This patch breaks PaStA if we don't use a patchwork configuration:

File "/media/Datenfass/PaStA/pypasta/Config.py", line 199, in init mbox_patchwork = mbox['patchwork'] KeyError: 'patchwork'

Please take care to not break the system. Only address patchwork-related config items, if a patchwork configuration is being used.

The same thing would happen if we removed [mbox.raw] or [mbox.pubin] from the Config. This PR assumes that the config will by default contain the line [mbox.patchwork], this is the same way as pubins or raw mboxes are handled

[rralf]

This patch breaks PaStA if we don't use a patchwork configuration: File "/media/Datenfass/PaStA/pypasta/Config.py", line 199, in init mbox_patchwork = mbox['patchwork'] KeyError: 'patchwork' Please take care to not break the system. Only address patchwork-related config items, if a patchwork configuration is being used.

The same thing would happen if we removed [mbox.raw] or [mbox.pubin] from the Config. This PR assumes that the config will by default contain the line [mbox.patchwork], this is the same way as pubins or raw mboxes are handled

Oh, does it? Thanks for the bug report! :+1: Will fix this...

[rsarky]

Please split it up into two patches: The first one to change load_index, the second one that pulls out the method.

I didnt pull out the load_index method. That was done by you in a previous commit. Are you talking about the load_mbox_file method by any chance?

[rsarky]

I wonder why the indentation is breaking. I used to use vim, but now I switched to vscode that uses pep8 style for formatting

[rsarky]

Hi,

before I continue with my review: please remove unnecessary if len() checks, and take care of the indentation. In all cases, you can remove the else statement. While it's surely not a good idea, there's no need to disallow raw mboxes in combination with patchwork. It's not required for our use-case, but there's no reason to forbid it.

Done

[rralf]

This patch breaks PaStA if we don't use a patchwork configuration: File "/media/Datenfass/PaStA/pypasta/Config.py", line 199, in init mbox_patchwork = mbox['patchwork'] KeyError: 'patchwork' Please take care to not break the system. Only address patchwork-related config items, if a patchwork configuration is being used.

The same thing would happen if we removed [mbox.raw] or [mbox.pubin] from the Config. This PR assumes that the config will by default contain the line [mbox.patchwork], this is the same way as pubins or raw mboxes are handled

Oh, does it? Thanks for the bug report! +1 Will fix this...

Aah, we do not have a bug here. We set default values in resource's common.cfg:

https://github.com/lfd/PaStA-resources/blob/master/common/default.cfg

So the fix is simple: In future, we must add an empty [mbox.patchwork] entry to default.cfg. But we can postpone that for the moment.

[rsarky]

This patch breaks PaStA if we don't use a patchwork configuration: File "/media/Datenfass/PaStA/pypasta/Config.py", line 199, in init mbox_patchwork = mbox['patchwork'] KeyError: 'patchwork' Please take care to not break the system. Only address patchwork-related config items, if a patchwork configuration is being used.

The same thing would happen if we removed [mbox.raw] or [mbox.pubin] from the Config. This PR assumes that the config will by default contain the line [mbox.patchwork], this is the same way as pubins or raw mboxes are handled

Oh, does it? Thanks for the bug report! +1 Will fix this...

Aah, we do not have a bug here. We set default values in resource's common.cfg:

https://github.com/lfd/PaStA-resources/blob/master/common/default.cfg

So the fix is simple: In future, we must add an empty [mbox.patchwork] entry to default.cfg. But we can postpone that for the moment.

Yup makes sense

[rralf]

Rebased your work and pushed it to wip-patchwork-integration. Will test it now.

[rsarky]

Hi,

just successfully run your script. There are still some issues, we receive duplicate messages. Let's talk about that in a short video conference. Will send you an email tomorrow.

Thanks Ralf

Sure

[rsarky]

Added some error handling. Tested it out by stopping the patchwork server while the update was running. Works!

Added it to this commit

[rralf]

Looks good now! Thanks, merged to next and wip-patchwork-integration.