2015 FRQ v. PBL Connections

[YLu-1258]

FRQ 1: Arrays and ArrayLists

The original FRQ for this topic was centered around creating a diverse array. While my project did employ the use of Arrays and ArrayLists, it was different. As previously mentioned, my role in our project is largely focused on developing and training the model. As such, I have to work with large quantities of data and break them down into smaller, more manageable chunks to use them for predictions. Below is an example of how arrays and ArrayLists were used in my code:

public ArrayList<Double> extractDataFromCSV() {
    ArrayList<Double> data = new ArrayList<Double>(); 
    try (BufferedReader br = new BufferedReader(new FileReader(file))) {
        String line = br.readLine(); // Used to be an array of strings, changed to just use a regular String object
        double closeValue;
        while ((line = br.readLine()) != null) {
            closeValue = Double.parseDouble(line.split(",")[featureIndex]);
            this.minClose = Math.min(closeValue, this.minClose);
            this.maxClose = Math.max(closeValue, this.maxClose);
            data.add(closeValue);
        }
        data = MinMaxScaler.minMaxScale(data, 0, 1);
        return data;
    } catch (IOException e) {
        e.printStackTrace();
        return null;
    }
}

I made the above function to extract our stock data from any given CSV file. In this case, I use an ArrayList of doubles to store the final output of the function as a plain Java ArrayList (so operations are easier down the line). With every iteration of my while loop, my function will extract the data, format it, and add it to the array. Additionally, an array is used, albeit implicity, in this function. The array appears in the following line:

closeValue = Double.parseDouble(line.split(",")[featureIndex]);

In this line, I split a string of values using a comma as a delimiter into a list. Through this method, I can access particular indices of the list, which represent the corresponding column of data that I want access to, granting me the ability to select my data

FRQ 2: Classes

In the original problem, the second FRQ was placed in the context of a word-guessing game (similar to wordle), where an object representing the word was created, along with attributes and methods to print out specific hints relevant to the word. In my project, I use classes to create objects denoting certain parts of the neural network. For instance, the dataset object I've created is an example of a class, with methods, attributes, and constructors.

@NoArgsConstructor
@AllArgsConstructor
@ToString
@Data
public class LSTMDataSetCreator {
    private String directory;
    private String ticker;
    private String file;
    private double splitRatio;
    private int features;
    private int labels;
    private int batchSize;
    private int stepCount;
    private int featureIndex;
    private double minClose;
    private double maxClose;

    public LSTMDataSetCreator(String directory, String ticker, double splitRatio, int features, int labels, int stepCount) {
        this.directory = directory;
        this.ticker = ticker;
        this.file = directory + "/" + this.ticker + "_test.csv";
        this.splitRatio = splitRatio;
        this.features = features; // 1
        this.labels = labels; // 1
        this.stepCount = stepCount; // 60
        this.featureIndex = 4;
        this.minClose = Double.MAX_VALUE;
        this.maxClose = Double.MIN_VALUE;
    }
   ... More methods and attributes below
}

Another example of a class that we used was our actual neural network, which in reality, was just a simple class with many attributes and a single method that created the structure of the network:

public class LSTMNetModel {
    private static final int seed = 12345;
    private static final int lstmLayer1Size = 32;
    private static final int lstmLayer2Size = 32;
    private static final int denseLayerSize = 32;
    private static final double dropoutRatio = 0.2;
    private static final int truncatedBPTTLength = 27;

    public static MultiLayerNetwork buildLstmNetworks(int nIn, int nOut) {
        MultiLayerConfiguration conf = new NeuralNetConfiguration.Builder()
                .seed(seed)
                .optimizationAlgo(OptimizationAlgorithm.STOCHASTIC_GRADIENT_DESCENT)
                .weightInit(WeightInit.XAVIER)
                .l2(1e-4)
                .list()
                .layer(0, new LSTM.Builder()
                        .nIn(nIn)
                        .nOut(lstmLayer1Size)
                        .activation(Activation.TANH)
                        .gateActivationFunction(Activation.HARDSIGMOID)
                        .dropOut(dropoutRatio)
                        .build())
                .layer(1, new LSTM.Builder()
                        .nIn(lstmLayer1Size)
                        .nOut(lstmLayer2Size)
                        .activation(Activation.TANH)
                        .gateActivationFunction(Activation.HARDSIGMOID)
                        .dropOut(dropoutRatio)
                        .build())
                .layer(2, new DenseLayer.Builder()
                    .nIn(lstmLayer2Size)
                    .nOut(denseLayerSize)
                .activation(Activation.RELU)
                .build())
                .layer(3, new RnnOutputLayer.Builder()
                        .nIn(denseLayerSize)
                        .nOut(nOut)
                        .activation(Activation.IDENTITY)
                        .lossFunction(LossFunctions.LossFunction.MSE)
                        .build())
                .backpropType(BackpropType.TruncatedBPTT)
                .tBPTTForwardLength(truncatedBPTTLength)
                .tBPTTBackwardLength(truncatedBPTTLength)
                .build();

        MultiLayerNetwork net = new MultiLayerNetwork(conf);
        net.init();
        net.setListeners(new ScoreIterationListener(100));
        return net;
    }
}

The main use of neural networks in my code was not just to create separate objects, but also for better functionality and organization. Having each component abstracted as an object allows me to use it like a tool in a toolbelt. Once I write the code for a particular feature, I'm not concerned with it as long as it works. Like how the word guessing class simplifies the hinting process, the classes I use allow my main code to be readable and functional.

FRQ 3: 2-D Arrays / Sparse Arrays

In our project, the most predominant form of a 2D Array was when we were creating the actual dataset for the training. Our Neural Network is an LSTM, which relies on something called a time series. Thus, a 2D array is required to represent both dimensions of our training data. The first dimension is the batch count of the data, which is how many training batches that we have. The second dimension is the batch size of the data, which is how many lines of data are within each training batch. As an example, our training dataset code is provided below:

public DataSet createTrainDataset() {
    ArrayList<Double> closeData = extractDataFromCSV();
    int maxIndex = (int) Math.ceil(closeData.size()*(this.splitRatio));
    double[][] XTrain = new double[maxIndex-60][60];
    double[] yTrain = new double[maxIndex-60];
    for (int i = 60; i < maxIndex; i++){
        for (int j = 0; j < this.stepCount; j++) {
            XTrain[i-60][j] = closeData.get(i-60+j);
        }
        yTrain[i-60] = closeData.get(i);
    }
    INDArray XTrainArray = Nd4j.create(XTrain); 
    INDArray yTrainArray = Nd4j.create(yTrain);
    DataSet trainingDataSet = new DataSet(XTrainArray, yTrainArray);
    System.out.println(trainingDataSet);
    return trainingDataSet;
}

As you can see, the XTrain variable is a 2D array of doubles. To simplify the code, we are taking increments of 60 days at a time and storing them as part of the training data, then following up with one additional day for the testing data. For each batch, we increment up one day, essentially incrementing through the entirety of our dataset day after day. However, our current data is converted into the INDArray object and is still two-dimensional. For this data to work, we must convert it into the three-dimensional format, with the third dimension being the time series, which would be a size of 1 for each input value. After doing that, we would have finished the second version of our LSTM model.

FRQ 4: Interfaces

The final FRQ of the practice exam was on Java interfaces. While I haven't used interfaces directly yet, they are present in our project as our StockJpaRepository interface. For what I can understand, a Java interface sort of acts like a blueprint for a class, denoting the methods that would need to be defined later on in the class definition. I sort of thought this as similar to a C/C++ header file, which is similar in nature with it's function declarations (but no definitions). Here is the code for our StockJpaRepository:

public interface StockJpaRepository extends JpaRepository<Stock, Long> {
    Stock findByName(String name);

    List<Stock> findAllByOrderByNameAsc();

    /* Custom JPA query articles, there are articles that show custom SQL as well
       https://springframework.guru/spring-data-jpa-query/
       https://docs.spring.io/spring-data/jpa/docs/current/reference/html/#jpa.query-methods
    */
    // Custom JPA query
    @Query(
            value = "SELECT * FROM Person p WHERE p.name LIKE ?1 or p.email LIKE ?1",
            nativeQuery = true)
    List<Stock> findByLikeTermNative(String term);
    /*
      https://www.baeldung.com/spring-data-jpa-query
    */
}

Obviously, this code was heavily borrowed from what Mr. Mort provided to us at the beginning of the year (but that's OK! We learn through emulating). This interface in particular allows for our APIs to perform certain CRUD operations on the database, such as READing certain stock data for users.

Overal FRQ Reflection

• FRQ 1: I believe this was the second most easiest FRQ. It was really simple for the most part, and the implementation was nothing new from what I have done before. The main take-away from this FRQ was using for-loops to iterate over array values, update them, or sum them for whatever functionality I required. Something that I do need to keep in mind is how to instantiate a new array, as I got used to the following syntax:

  type[] array = new type[length];

• FRQ 2: This FRQ was slightly harder than the last one, but it was no trouble either. Once again, I used a for-loop to iterate over string that was provided to me. I was careful to note that a string was really just a collection of characters, so reading the string this way would mean that I am working with the character type, and not the string type. One other important part in this FRQ was sequential order. It mattered in what order I checked the "correctness" of the letter, as the program would only work if it checked for the right position first, then existence but wrong position, then finally wrong character. Otherwise, it would incorrectly flag the character as being in the wrong position, when it really was in the right position
• FRQ 3: This FRQ was probably the hardest of the bunch, and that had to do with my implementation of the code. You see, I used a for-loop to iterate over and delete elements in the array, which in programming, is a huge taboo. Doing so would mess up the indicie of the array, making a simple for-loop an incorrect approach to deleting values in the array. But since I already attempted this solution, I sought to make it work. Instead of actively deleting the values in the array, I simply copied their indices into a new arraylist of values, which I then iterated over to delete. Following this, I was quick to update the values of the remaining elements, as the order of the array shifted once more. After that, the solution worked. However, I do recognize that using a while loop, and checking for the existence of the next element in the array would've made for a much cleaner solution.
• FRQ 4: This FRQ was probably the easiest, as all I had to do was create an interface and show that I knew how to use it. After I created the interface, I just needed to use the @Override annotation to define the methods for my classes, and that was pretty much it.

Links

Title	Link
FRQ 1	link
FRQ 2	link
FRQ 3	link
FRQ 4	link

[jm1021]

This is exceptional work. Finally have someone that is using 2D arrays!. The use case being you have with primitive double, fits CB perfectly.

On Class FRQ, you through me into a 2 hour detour tonight thinking about chaining and the "builder pattern". I decided to make my own code to use it here. Here is sample.

       FruitBasket basket = new FruitBasket();
        // Add fruits to the collection using chaining
        basket
            .addFruit(new Fruit("Apple", Climate.TEMPERATE))
            .addFruit(new Fruit("Banana", Climate.TROPICAL))
            .addFruit(new Fruit("Cherry", Climate.TEMPERATE))
            .addFruit(new Fruit("Orange", Climate.SUBTROPICAL))
            .addFruit(new Fruit("Peach", Climate.SUBTROPICAL))
            .addFruit(new Fruit("Pear", Climate.TEMPERATE))
            .addFruit(new Fruit("Pineapple", Climate.TROPICAL))
            ;

It is cool thing to know/code as your library and spring security use it. This is the rage in new Java, way past CB usage of prre Java 8.

I enjoyed reading your journey and reflections. All of it is very "engineering"!!!

[drewreed2005]

Grader: Drew Reed

Crossover Grading for FRQ/PBL

FRQs

Title	Description	Score
FRQ 1	All code is shown working without unnecessary complexity. Reflections are concise and give insight into the thought process behind the problem. However, they could be more fleshed-out. The class is in a fully functional state.	0.95/1
FRQ 2	All code is shown working without unnecessary complexity. There was good thought put into abstracting the process of searching the word for a character into a separate method. The reflection does a good job pointing out the important steps of the process; I like that it points out the importance of character order. Again, reflection could have gone more into depth on personal thoughts (though it's not a difficult problem). The class is in a fully functional state.	0.95/1
FRQ 3	All code is shown working without unnecessary complexity. Once again, abstracting some more complicated processes into separate classes was a decision that showed consideration into the readability and complexity of the code. Solution is thoughtful and recognizes the importance of retaining order in the ArrayList. This made me think about my own solution and ways to improve it, even without a setter, using the ArrayList set method. The class is in a fully functional state.	0.98/1
FRQ 4	All code is shown working without unnecessary complexity. Broader reflection or something, even though the problem was very easy and likely took not much thought. Full scope of classes shown working together with thought put into utilizing the classes' contains methods.	0.97/1
Total	(1.125 0.9) (0.95 + 0.95 + 0.98 + 0.97)	3.898125/3.6

PBL Reflection

Title	Description	Score
FRQ 1	ArrayLists clearly have a strong presence in your project and its code. Your discussion of iteration through a while loop shows consideration of the contents of this specific FRQ problem and its teachings, as well as your decision to show a more minimal use of an array in your program. You pretty much effortlessly meet these standards.	0.98/1
FRQ 2	You do a good job showing the integration of classes with each other to work toward the functionality of your program. It would be a bit easier to understand these connections if you were to explain the purpose for their connections more explicitly and simply. Aside from this, you clearly meet the requirements within your project and show good consideration of the contents of the FRQ question with your reflection.	0.95/1
FRQ 3	I really like your reflection here. You take a complicated segment of your functional code and explain not just the presence of 2D arrays within them, but also the broader purpose for using that data type. The only potential improvement on this reflection would include more consideration toward the actual contents of the FRQ, being mainly focused on the presence of a Sparse Array model.	0.97/1
FRQ 4	Yippee! JPA Repository interface mentioned! While I really like that you recognize the presence of interfaces in your project, I think you could have shown a bit more for this one to represent the depth of the Methods and Control Structures-type problems. Overall, though, you do a solid job here.	0.95/1
Total	0.9 * (0.98 + 0.95 + 0.97 + 0.95)	3.465/3.6

Your overall reflection fills in the blanks of a lot of the areas where I think your FRQ reflections were slightly lacking. Considering your notes on each question, it's clear that you put a lot of thought into the contents of each question and their connections to your project. Great work.

[jm1021]

Crossover Grading for FRQ/PBL

FRQs

Title Description Score FRQ 1 All code is shown working without unnecessary complexity. Reflections are concise and give insight into the thought process behind the problem. However, they could be more fleshed-out. The class is in a fully functional state. 0.95/1 FRQ 2 All code is shown working without unnecessary complexity. There was good thought put into abstracting the process of searching the word for a character into a separate method. The reflection does a good job pointing out the important steps of the process; I like that it points out the importance of character order. Again, reflection could have gone more into depth on personal thoughts (though it's not a difficult problem). The class is in a fully functional state. 0.95/1 FRQ 3 All code is shown working without unnecessary complexity. Once again, abstracting some more complicated processes into separate classes was a decision that showed consideration into the readability and complexity of the code. Solution is thoughtful and recognizes the importance of retaining order in the ArrayList. This made me think about my own solution and ways to improve it, even without a setter, using the ArrayList set method. The class is in a fully functional state. 0.98/1 FRQ 4 All code is shown working without unnecessary complexity. Broader reflection or something, even though the problem was very easy and likely took not much thought. Full scope of classes shown working together with thought put into utilizing the classes' contains methods. 0.97/1 Total (1.125 0.9) (0.95 + 0.95 + 0.98 + 0.97) 3.898125/3.6

PBL Reflection

Title Description Score FRQ 1 ArrayLists clearly have a strong presence in your project and its code. Your discussion of iteration through a while loop shows consideration of the contents of this specific FRQ problem and its teachings, as well as your decision to show a more minimal use of an array in your program. You pretty much effortlessly meet these standards. 0.98/1 FRQ 2 You do a good job showing the integration of classes with each other to work toward the functionality of your program. It would be a bit easier to understand these connections if you were to explain the purpose for their connections more explicitly and simply. Aside from this, you clearly meet the requirements within your project and show good consideration of the contents of the FRQ question with your reflection. 0.95/1 FRQ 3 I really like your reflection here. You take a complicated segment of your functional code and explain not just the presence of 2D arrays within them, but also the broader purpose for using that data type. The only potential improvement on this reflection would include more consideration toward the actual contents of the FRQ, being mainly focused on the presence of a Sparse Array model. 0.97/1 FRQ 4 Yippee! JPA Repository interface mentioned! While I really like that you recognize the presence of interfaces in your project, I think you could have shown a bit more for this one to represent the depth of the Methods and Control Structures-type problems. Overall, though, you do a solid job here. 0.95/1 Total 0.9 * (0.98 + 0.95 + 0.97 + 0.95) 3.465/3.6 Your overall reflection fills in the blanks of a lot of the areas where I think your FRQ reflections were slightly lacking. Considering your notes on each question, it's clear that you put a lot of thought into the contents of each question and their connections to your project. Great work.

This is outstanding two part evaluation. The grader looked for depth and used their experience in each part of the FRQ and PBL/FRQ evaluations. I see the grader being vested and technically involved, bring in their own experience. Tone may be a bit of a concern, but this depends on collaborative relationships. Sometimes this direct tone can be fantastic.

There are very clear on mathematics as well, which means they are consider Teacher will want to see what is written. Only flaw is Alex gets full multiple on both parts.