UX refactoring proposal

[alexcombessie]

Hi @bgalego,

Please find my pull request ready for review. All in all the plugin was working well, so I have not changed the logic of the code but focused on refining the UX.

Here is a summary of the main changes.

• Simplified UX: reordered and reduced the number of parameters for novice users, added links to the doc on the preset, standardized output column names (no uppercase to avoid issues with Hive)
• Code styling: applied black code formatting for better readability
• Language detection: both language code and name are now in the output by default

• Sentiment analysis: added subjectivity, irony, agreement and confidence scores

  If that's OK with you, we can publish this new version to the Dataiku plugin store.

For future versions, I have noted a few ideas for enhancement you could look into:

1. Parallelize your code to make it run faster and improve API error handling. You can reuse the same framework I have built for other plugins. In a nutshell, you can
   • parallelize your API calling function without formatting, just return the response as JSON, using api_parallelizer.py
   • write CustomAPIFormatter classes for each API endpoint to turn your JSON response into nicely formatted new columns, similarly to SentimentAnalysisAPIFormatter
   • expose parallelization parameters in the preset for admins to tune it With this change, you could make your recipes run at least 4 times faster.
2. Allow for multiple languages in a single dataset. You can use the language code column as a parameter to the recipes.
3. Add entity and concept sentiment to the Sentiment Analysis recipe
4. Add all of your APIs to the plugin, which means adding:
   • Lemmatization, PoS and Parsing
   • Corporate Reputation
   • Text Clustering
   • Document Structure Analysis

Thanks again for the good work and drive to make this plugin available to Dataiku users.

Best regards,

Alex

[alexcombessie]

@bgalego thanks! I have added your suggested changes. Let me know if you have any additional suggestions!

[bgalego]

That's great @alexcombessie! I'm half-way through the review (I'm also updating our documentation at the same time), but everything looks great!

[alexcombessie]

Hi @bgalego! Thanks for the time spent this week to refine this PR. If the plugin doc is ready, do you think we can publish next week? Cheers, Alex

[bgalego]

I think so! I'll do a quick once-over on Monday, but I think everything will be fine. I also was thinking of increasing the version number to 1.0.1, but I haven't checked yet if it makes sense directly in the PR, or right after when I release a new version in the master branch.

I'll get back to you on Monday. Thanks for all the hard work!