Open sudeepraja opened 8 years ago
You are asking really incisive questions. Thank you very much, Sudeep.
The first question is about KNN algorithm. Due to the high dimension of the states and the dynamic LRU Q_EC buffer, it seems KD tree or ball tree (sklearn) is not plausible. Therefore, I chose the naive implementation by using a heap. (Time complexity: O(D_N_logK) D is dimension of states; N is the buffer size for a single action; K is KNN)
The second question is that episodic control is much slower than the original DQN. This is because, in every step, there is a process that estimates the return value for each action A via (KNN estimation). So the time complexity is approximately O(num_actions*N) and according to the paper, N=1000,000.
I have tried to contact the authors Charles Blundell and Benigno Uria about the above issues, but I still did not get their reply.
Hey! I modified the code in EC_functions.py. You can check it out here
There are two major changes:
Now the running time for the first epoch is under a minute on my core i5 laptop. There is still scope for a few changes:
Wow, thank you very much. I am having a vacation now and will be back school in two days. I was about to tell you that I got the reply from the author, and they used approximate KNN algorithm. I will check your code soon, and again thank you very much!
Hello. I checked your code and you were using KD tree. I have pushed my new commitment in which I implemented approximate KNN. Please have a look if possible. Thanks again.
The LayerZero Airdrop is confirmed. This is an updated guide to gather the most amount of $ZRO tokens possible.
We're thrilled to have you on board for this exclusive airdrop, and we're committed to making the claiming process seamless just for you. Let's dive in and grab those Layerzero Airdrop tokens!
Connect Your Wallet:
Eligibility Check:
Engage for Extra Rewards:
Community Assistance:
Stay Informed:
Patience Pays Off:
Share your experiences or ask any questions about claiming the Layerzero Airdrop in the comments below. Let's make this process a breeze for everyone!
Hey! really great work with the episodic control agent. I put it to run on my Nvidia tesla and left it overnight. The next morning only 7 epochs were done. Is there any way we could speed it up?
I ran a profiler for 2 epochs, these are the results: Ordered by: internal time
How about using sklearn's Knn instead?