search

zhuyitan / IGTD

Image Generator for Tabular Data (IGTD): Converting Tabular Data to Images for Deep Learning Using Convolutional Neural Networks
MIT License
157 stars 59 forks source link

Time complexity is high when column number arbitrary in image transformation #9

Closed ofakduman closed 1 year ago

ofakduman commented 1 year ago

image When using the image transformation, depending on the number of columns (for example ~20 thousand colons), I encounter quite long algorithm run times, for example about 30 hours, but as a result, I get a crash. I got around this by reducing the number of columns. What are your suggestions, what would you recommend? Thank in advance

zhuyitan commented 1 year ago

If the number of features is huge, IGTD takes a very long time to convert data into images. I would suggest performing some feature selection analysis to reduce the feature number before transformation to image.

There will be a new version of the IGTD that uses multiple processes to optimize and reduces the time for generating images. But the release date will depend on development progress and is not set yet.

Thanks for your interest in using IGTD.

Yitan

On Mon, Dec 5, 2022 at 6:57 AM Ömer Faruk Akduman @.***> wrote:

[image: image] https://user-images.githubusercontent.com/61577706/205641198-62503107-bf8a-4b08-9297-3b15321d1184.png When using the image transformation, depending on the number of columns (for example ~20 thousand colons), I encounter quite long algorithm run times, for example about 30 hours, but as a result, I get a crash. I got around this by reducing the number of columns. What are your suggestions, what would you recommend? Thank in advance

— Reply to this email directly, view it on GitHub https://github.com/zhuyitan/IGTD/issues/9, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACOHXUHM4GWE57FPCUYED5DWLXRD5ANCNFSM6AAAAAASUHYF4Q . You are receiving this because you are subscribed to this thread.Message ID: @.***>

ofakduman commented 1 year ago

If the number of features is huge, IGTD takes a very long time to convert data into images. I would suggest performing some feature selection analysis to reduce the feature number before transformation to image. There will be a new version of the IGTD that uses multiple processes to optimize and reduces the time for generating images. But the release date will depend on development progress and is not set yet. Thanks for your interest in using IGTD. Yitan … On Mon, Dec 5, 2022 at 6:57 AM Ömer Faruk Akduman @.> wrote: [image: image] https://user-images.githubusercontent.com/61577706/205641198-62503107-bf8a-4b08-9297-3b15321d1184.png When using the image transformation, depending on the number of columns (for example ~20 thousand colons), I encounter quite long algorithm run times, for example about 30 hours, but as a result, I get a crash. I got around this by reducing the number of columns. What are your suggestions, what would you recommend? Thank in advance — Reply to this email directly, view it on GitHub <#9>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACOHXUHM4GWE57FPCUYED5DWLXRD5ANCNFSM6AAAAAASUHYF4Q . You are receiving this because you are subscribed to this thread.Message ID: @.>

Thank you for your suggest, I am waiting for the new version of the igtd algorithm. Yours truly