Loading Network Problems

[jordanott]

Do additional parameters need to be set after loading a network? In test_network_save.f90 weights and biases are only shown to be equal. Could you provide an example to load a network then evaluate its accuracy?

I'm able to load previously saved networks. However, when attempting to train them or evaluate them for accuracy I encounter segfaults.

program example_simple
  use mod_network, only: network_type
  implicit none
  type(network_type) :: net, net2
  real, allocatable :: input(:), output(:)
  integer :: i
  net = network_type([3, 5, 2])

  ! Saving network 
  call net % save('my_simple_net.txt')

  ! Loading network 
  call net2 % load('my_simple_net.txt')

  input = [0.2, 0.4, 0.6]
  output = [0.123456, 0.246802]

  do i = 1, 500
    ! Segfault training net2
    call net2 % train(input, output, eta=1.0)
    print *, 'Iteration: ', i, 'Output:', net2 % output(input)
  end do
end program example_simple

[milancurcic]

Hi Jordan,

I could reproduce this. I tracked the segfault to activation functions procedure pointers not being set when loading a network from file.

The simple workaround in your example is to explicitly set the activation function immediately following the net % load():

call net2 % load('my_simple_net.txt')
call net2 % set_activation('sigmoid')

This issue was most likely introduced with a recent refactor that allows activation functions to be set on a per-layer basis. Proper testing should have caught this, so your issue also raises a need for better testing.

However, it also raises a bigger issue -- user shouldn't have to keep track of which activation functions have been used, and this is even more important if saved network files are passed between users. The activation functions (and other metadata) should be properly saved and loaded via net % save() and net % load(), respectively.

Finally, here's an example of what you asked for. I also added this example in the 12-loading-network-problems branch.

program example_save_and_load

  use mod_network, only: network_type
  implicit none

  type(network_type) :: net1, net2
  real, allocatable :: input(:), output(:)
  integer :: i

  net1 = network_type([3, 5, 2])

  input = [0.2, 0.4, 0.6]
  output = [0.123456, 0.246802]

  ! train network 1
  do i = 1, 500
    call net1 % train(input, output, eta=1.0)
  end do

  ! save network 1 to file
  call net1 % save('my_simple_net.txt')

  ! load network 2 from file 
  call net2 % load('my_simple_net.txt')
  call net2 % set_activation('sigmoid')

  print *, 'Network 1 output: ', net1 % output(input)
  print *, 'Network 2 output: ', net2 % output(input)
  print *, 'Outputs match: ', all(net1 % output(input) == net2 % output(input))

end program example_save_and_load

And here's example output:

$ ./example_save_and_load 
 Network 1 output:   0.123456128      0.246801987    
 Network 2 output:   0.123456128      0.246801987    
 Outputs match:  T

[milancurcic]

Hi Jordan, I believe https://github.com/modern-fortran/neural-fortran/pull/17 resolves this issue. Please close if you agree or let me know otherwise.