search

pachoning / bigmds

Other
16 stars 2 forks source link

User provide the distance matrix #5

Open Antguz opened 5 days ago

Antguz commented 5 days ago

Hi,

I have found this package really useful and fast!

I have seen a request #4 where previously user can select the type of distance to estimate. I am wondering if there is any chance to implement an even broader functionality where users provide the distance matrix before hand?

Is this possible?

AG.

pachoning commented 4 days ago

Hi @Antguz , I expect to address the distance function really soon. Regarding providing the distance matrix, how would you provide it? If the distance matrix is large, it would be impossible to store it in a R object. If it is possible, then, you can use the cmdscale (or any other) function to perform MDS. Any idea on that?

Thanks!

Antguz commented 4 days ago

Hi @pachoning - thanks for your quick response.

In my case, for instance, my distance matrix is on the limit of cmdscale but it still takes two days to run; thus, an functionality when we provide a medium size distance matrices will definitely helps to overcome this processing time.

In addition, the storage size of the distance matrices depends on many factors (i.e., number of decimals, passing floats to integers by scaling, ....). Therefore, it should not limit the goal of a function.

Please consider also that, in research, big data and big matrices are also associated with big infrastructure like High Performance Computing (HPC). Virtual memory of ~2TB are common to see in HPC from many centers. So, an application like I mentioned could have a broader impact and likely to be upscale it.

Before my question, I was trying to modify your code. I noticed that a basic distance matrix (i.e., x_dis) (_n x n_) can be provided instead of x and it can be applied to some of the mds methods. Do you think that something like could be implemented?

AG.