Question: custom network for Ai deck (classification example)

[godhj93]

Hi all,

I am trying to upload my custom network to the ai-deck, but it has a problem.

my network structure is as follows:

class Encoder_tf(tf.keras.Model):

    def __init__(self, obs_shape):
        super().__init__()

        assert len(obs_shape) == 3

        self.repr_dim = 32 * 35 * 35 +(7*3)

        self.convnet = tf.keras.Sequential([
            tf.keras.layers.Conv2D(filters = 32, kernel_size = 3, strides = 2, activation='relu'),
            tf.keras.layers.Conv2D(filters = 32, kernel_size = 3, strides = 1, activation='relu'),
            tf.keras.layers.Conv2D(filters = 32, kernel_size = 3, strides = 1, activation='relu'),
            tf.keras.layers.Conv2D(filters = 32, kernel_size = 3, strides = 1, activation='relu'),
        ])     

        self.flatten = tf.keras.layers.Flatten()
#         self.concat = tf.keras.layers.Concatenate()
        self.linear = tf.keras.layers.Dense(10)
        self.linear2 = tf.keras.layers.Dense(10)
        self.globalpool = tf.keras.layers.GlobalAveragePooling2D()

    def call(self, x):
        #Input: 64x64x4
        (obs, drone_states) = x

        obs = obs / 255.0 - 0.5

        h = self.convnet(obs)
        h = self.globalpool(h)
        h = self.linear(h)

        drone_states = self.flatten(drone_states) # 9*9*32 = 2592
        drone_states = self.linear2(drone_states)

        h = tf.math.add(h, drone_states)

        return h

and whenever add or concatenation operation exists, the following docker command does not work.

docker run --rm -v ${PWD}:/module aideck-with-autotiler tools/build/make-example examples/ai/classification clean model build image

and I checked working well in two cases

#CASE 1
  def call(self, x):
        #Input: 64x64x4
        (obs, drone_states) = x

        obs = obs / 255.0 - 0.5

        h = self.convnet(obs)
        h = self.globalpool(h)
        h = self.linear(h)        
        return h

#CASE 2
  def call(self, x):
        #Input: 64x64x4
        (obs, drone_states) = x
        drone_states = self.flatten(drone_states) # 9*9*32 = 2592
        drone_states = self.linear2(drone_states)
        return drone_states

but when the two features (drone_states and h) meets using concatenation or add(I fit the shape of the features to work) it fails to build it.

the error is on https://godhj93.notion.site/github-issue-2-6a9a9dcaeb844a7e8ac96a94a74febdc

Is there any possible solution to work concatenation or add operation?

[hmllr]

Hi, this seems to be a question about the autotiler/NNtool, not really about the AI-deck - so I'd suggest you ask it there (https://github.com/GreenWaves-Technologies/gap_sdk/issues) (what you apparently already did). Maybe one thing to add is that the docker gap-sdk version is not the newest, so updating might help you - however, we did not test to run the existing examples with the newest sdk. Good luck with this and I am sorry to not be of much help on this topic, Hanna

[godhj93]

@hmllr It is solved, Thank you for helping :)

[hmllr]

Nice! Could you please point out here how you solved it for future readers encountering the same issue? That would be amazing :)

[godhj93]

Sure :)

It was very tiny problem. As I mentioned above, my network gets two inputs as follows:

(obs, drone_states) = x

But I did not edit a source code in "classification.c" to make the network get the inputs.

So I changed the source code in "classification.c" from

static void RunNetwork()
{
  __PREFIX(CNN)
  (cameraBuffer, Output_1);
}

to

static void RunNetwork()
{
  __PREFIX(CNN)
  (cameraBuffer, Input2, Output_1);
}

For anyone who has the problem like me, my solution will be helpful :)