[Question]: Issue implementing a model using a TextVectorization layer

[BlackholeGH]

Description

Hello, I am crossposting this query from Stack Overflow, as I imagine there may be some more bespoke advice available here. Apologies for what is likely a beginner question.

I am trying to learn the API for Tensorflow.NET and Keras.NET from SciSharp to do some machine learning work in C#. I have some familiarity with Tensorflow and Keras in Python, but am by no means an expert. I am trying to adapt Keras documentation for Python into the syntax of the .NET implementation, and so it is difficult to find concrete examples for my exact case.

At present, I am simply trying to build a simple model that attaches a TextVectorization layer that takes a string input to an Embedding layer, which would then output a vector. I have a text corpus that I am looking to adapt the Vectorizer to, and then I am hoping to process text tokens in batches of 12. I have made multiple attempts using both the functional and sequential APIs and have yet to get it to work with either.

My current code looks like this, but this is after shifting and moving things around several times to try this in different ways. Each time I am getting a different error at runtime:

    internal class Program
    {
        static void Main(string[] args)
        {
            String[] TED = File.ReadAllText("ted_unf.txt").Split('\n');
            var io = DataUtil.TokenizeAndEmbed(TED);
            while(true)
            {
                String[] testdata = new String[12];
                for(int i = 0; i < 12; i++)
                {
                    if (testdata[i] == null)
                    {
                        testdata[i] = "Total";
                    }
                }
                TED[0].Split(' ').Take(12).ToArray().CopyTo(testdata, 0);
                var cdat = tf.constant(testdata[0], shape: (1));
                object output = io.predict(tf.constant(testdata));
                Console.WriteLine(output);
                Console.ReadLine();
            }
        }
    }
    public static class DataUtil
    {
        public static Tensorflow.Keras.Engine.Model TokenizeAndEmbed(string[] texts)
        {
            var In = tf.keras.Input(shape: (1, 12), dtype: tf.@string);
            Tensor inTexts = tf.constant(texts);
            var Tokenize = tf.keras.layers.preprocessing.TextVectorization(split: "whitespace", max_tokens: 9999, output_mode: "int", output_sequence_length: 12);
            Tokenize.adapt(inTexts);
            var Embed = tf.keras.layers.Embedding(10000, 64);
            var mod = keras.Sequential();
            //mod.add(In);
            mod.add(Tokenize);
            mod.add(Embed);
            return mod;
        }
    }

Note that I have an explicit input layer that is currently commented out from being added to the model. The Keras documentation for TextVectorizeration gives an example which says that an explicit input layer is required (https://keras.io/api/layers/preprocessing_layers/text/text_vectorization/), whereas other sources state that it is only required when using the functional API. Regardless, when I try to add it with either API in Keras.NET, I get an error when building the model object:

The tensor input_1:0 at layer text_vectorization is part of a cycle.

This occurs when attempting to add the layer "Tokenize" to the model.

I understand that a cycle means that the layer is somehow dependent on itself, but I am unsure how/why this would be the case. Regardless, I can omit the explicit input layer and the model will be created, but then attempting to query it gives me a nullreferenceexeption at io.predict:

   Object reference not set to an instance of an object.
   at Tensorflow.Keras.Engine.Sequential._build_graph_network_for_inferred_shape(Shape input_shape, TF_DataType input_dtype)
   at Tensorflow.Keras.Engine.Sequential.Call(Tensors inputs, Tensor state, Nullable`1 training)
   at Tensorflow.Keras.Engine.Layer.Apply(Tensors inputs, Tensor state, Boolean training)
   at Tensorflow.Keras.Engine.Model.predict_step(Tensors data)
   at Tensorflow.Keras.Engine.Model.run_predict_step(OwnedIterator iterator)
   at Tensorflow.Keras.Engine.Model.PredictInternal(DataHandler data_handler, Int32 verbose)
   at Tensorflow.Keras.Engine.Model.predict(Tensors x, Int32 batch_size, Int32 verbose, Int32 steps, Int32 max_queue_size, Int32 workers, Boolean use_multiprocessing)
   at TransformerModule_One.Program.Main(String[] args) in G:\Documents\Projects\Programming\Visual Studio\source\repos\TransformerModule-One\TransformerModule-One\Program.cs:line 34

I'm not sure if this is an issue with the model setup or if I am feeding it input data in the wrong shape or format... I am not sure how I can control the format without the explicit input layer, or what exactly the TextVectorization layer is expecting as input other than it should be a tensor containing strings.

Anyway, I would appreciate any assistance in understanding what's going wrong here and how to correctly approach this.

Thank you kindly.

Alternatives

No response

[BlackholeGH]

I'm closing this after further investigation, as I don't think TensorFlow.NET is best suited to my use-case. I think I had some confusion between Keras.NET and Tensorflow.Keras; I will be using the former with Python bindings.

[AsakusaRinne]

Hi, thank you for reporting us this issue. We were busy on refactoring some codes and adding new features that we didn't manage to make quick reply to issues.

The main difference between Keras.NET and Tensorflow.Keras is that Keras.NET requires python environment. Also, the two repos have different maintainers from SciSharp community.

We'll fix this issue in the future after refactoring. Therefore we'll re-open this issue and I think we won't interrupt you since you've given an example to reproduce it. Good luck with Keras.NET!