D3_FORMATS PROBLEM

[iRoot13x18]

Bug description

When I change the configuration of the superset_config.py file and I want to change the number format to European when I reboot to apply the changes nothing comes up.

How to reproduce the bug

1. go to superset_config.py or config.py

2. add or edit this lines

   • D3_FORMATS = {

   'NUMBER': {

   'thousands': '.',
   
   'decimal': ','

   },

Screenshots/recordings

No response

Superset version

master / latest-dev

Python version

3.9

Node version

16

Browser

Chrome

Additional context

No response

Checklist

• [X] I have searched Superset docs and Slack and didn't find a solution to my problem.
• [X] I have searched the GitHub issue tracker and didn't find a similar bug report.
• [X] I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.

[dosubot[bot]]

To change the number format to European in Apache Superset, modifying the superset_config.py file alone might not be sufficient. Instead, you should update the D3FormatConfig.ts file in the frontend code. Here’s how you can do it:

1. Locate the D3FormatConfig.ts file in the frontend codebase:

   superset-frontend/packages/superset-ui-core/src/number-format/D3FormatConfig.ts

2. Update the DEFAULT_D3_FORMAT parameter to use a European format locale definition. Here’s an example:

   import { FormatLocaleDefinition } from 'd3-format';
   
   export const DEFAULT_D3_FORMAT: FormatLocaleDefinition = {
    decimal: ',',
    thousands: '.',
    grouping: [3],
    currency: ['€', ''],
   };

This change will ensure that the number format is applied globally across your Superset instance ^[1].

Additionally, ensure that there are no overriding configurations in other files such as superset_config_docker.py or any other local configuration files that might affect the D3_FORMATS settings ^[2][3][4].

To continue talking to Dosu, mention @dosu.

[rusackas]

D3_FORMATS isn't a thing, as far as I know. There is D3_FORMAT in config.py, but that takes the parameters for a single format, not an array or object of various formats like you seem to be using.

[fabgat]

Hi, I've modified the D3FormatConfig.ts file, as per follow: import { FormatLocaleDefinition } from 'd3-format';

export const DEFAULT_D3_FORMAT: FormatLocaleDefinition = { decimal: ',', thousands: '.', grouping: [3], currency: ['€', ''], };

I've modified also the config.py:

class D3Format(TypedDict, total=False): decimal: str thousands: str grouping: list[int] currency: list[str]

D3_FORMAT: D3Format = {}

D3_FORMAT: D3Format = { "decimal": ",", "thousands": ".", "grouping": [3], "currency": ["€", ""] }

no changes into: superset_config.py

After those activities I've restarted the docker, but everything is the same, no changes.

Where I'm wrong?

Regards

[rusackas]

@dosu-bot

[dosubot[bot]]

To ensure that your changes to D3FormatConfig.ts and config.py are applied correctly, follow these steps:

1. Recompile the Frontend Code: After modifying D3FormatConfig.ts, you need to recompile the frontend assets. You can do this using Docker commands provided in the Superset repository:

   docker-compose up

   Alternatively, you can use the js_build.sh script located in the superset-frontend directory to rebuild the frontend assets:

   cd superset-frontend
   ./js_build.sh

2. Ensure Configuration is Loaded: Make sure that your custom superset_config.py is recognized by setting the SUPERSET_CONFIG_PATH environment variable to the full path of your superset_config.py file. For example:

   export SUPERSET_CONFIG_PATH=/app/superset_config.py

   If you are using Docker, you can add your overrides as shown below:

   COPY --chown=superset superset_config.py /app/
   ENV SUPERSET_CONFIG_PATH /app/superset_config.py

3. Restart Services in Correct Order: Restart the services in the following order to ensure that the configuration changes are applied correctly:

   docker-compose restart redis db superset superset-init superset-worker superset-worker-beat

4. Clear Cache: Clear the cache to ensure that the new configuration is applied. You can do this by adding the following lines to your superset_config.py:

   from superset.utils.cache_manager import cache_manager
   
   cache_manager.cache.clear()
   cache_manager.data_cache.clear()
   cache_manager.thumbnail_cache.clear()
   cache_manager.filter_state_cache.clear()
   cache_manager.explore_form_data_cache.clear()

   Alternatively, you can clear the cache manually using the following methods:

   cache_manager.cache.clear()
   cache_manager.data_cache.clear()
   cache_manager.thumbnail_cache.clear()
   cache_manager.filter_state_cache.clear()
   cache_manager.explore_form_data_cache.clear()

By following these steps, you should be able to apply the changes to the number format configuration successfully ^[1][2][3][4].

To continue talking to Dosu, mention @dosu.

[fabgat]

Thanks a lot. Unfortunately also following your instructions, I'm not able to see deployed Italian D3. I still see everything as before, even into drop down menu into charts menu, where I can choose a different "number formatting", into the drop down menu, I still see the original once.

[fabgat]

At the end, I've D3-format working.

Those are my 50 cents.

Superset Version: 4.0.0 Docker version 26.1.1, build 4cf5afa

To whom it may concern, here follows the two steps needed: 1) update the file: superset-frontend/src/setup/setupFormatters.ts into that file, I've added the needed configuration:

.registerValue('CURRENCY_ITALY', createD3NumberFormatter({
  locale: {
    decimal: ',',
    thousands: '.',
    grouping: [3],
    currency: ['€', ''],
  },
  formatString: '€.,2f',
}))

after changing it, I proceed to build the frontend: npm run build

D3_FORMAT = { "decimal": ",", "thousands": ".", "currency": ["€", ""], "grouping": [3], }

After this I've build the new docker image as follow:

sudo docker build -t my-custom-superset-image . <--- pay attention to not forget the "." at the end.

Now I got the choice into the dropdown menu into charts. That's all.

Hope it helps.

[fabgat]

Now I'm trying to apply same number formatting to the export feature. I've first added this into superset_config.py: CSV_EXPORT = {"encoding": "utf-8", "sep": ";", "decimal": ",", "thousands": "."}

But when I try to download dashboard in CSV format, I receive this log: NDFrame.to_csv() got an unexpected keyword argument 'thousands'

So I have removed "thousands" from superset_config.py and launched again superset. Trying new conf, now I can download the csv but I only see that the new separator is: ":".

Decimal remained as ".".

Any ideas how to implement: decimal , thousands .

Thanks

[sfirke]

There are many things going on in this issue, much of which have been resolved. I'm going to close it because that makes issues not useful for bug tracking. To recap + respond:

• How to change the comma separator of how Superset displays float numbers: people should be able to simply set D3_FORMAT= in their custom superset_config.py as covered in this discussion and the pull request it links to: https://github.com/apache/superset/discussions/29113 You should not have to change Superset's code and rebuild an image. If this isn't working please post in that discussion comments or on Slack.
  • The original poster of this GitHub issue had a problem caused by a typo, pluralizing the variable name.
• Changing the format of the export via CSV_EXPORT: @fabgat I agree with your approach of passing argument to pandas DataFrame.to_csv via the CSV_EXPORT config settings. Looks like thousands is not a supported keyword, see the docs here: https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.to_csv.html I think your best bet is to investigate the float_format argument, playing with it outside of Superset until you find the setting you need.

If you think this needs work, is a bug, or you have a contribution to make, feel free to post a new GitHub issue and tag me. Or if you get it working, feel free to reply here with your solution or make a new GitHub Discussion and self-answer it so others can learn from it.