Once upon a Time on Neural Coding.

[NorbertZheng]

Related Reference

• Wu S, Nakahara H, Murata N, et al. Population decoding based on an unfaithful model.
• Wu S, Nakahara H, Amari S. Population coding with correlation and an unfaithful model.
• Wu S, Amari S. Neural implementation of Bayesian inference in population codes.
• Oizumi M, Ishii T, Ishibashi K, et al. Mismatched decoding in the brain.
• Xiao L, Zhang M, Xing D, et al. Shifted encoding strategy in retinal luminance adaptation: from firing rate to neural correlation.
• Dong X, Ji Z, Chu T, et al. Adaptation Accelerating Sampling-based Bayesian Inference in Attractor Neural Networks.
• Hoyer P, Hyvärinen A. Interpreting neural response variability as Monte Carlo sampling of the posterior.
• Echeveste R, Aitchison L, Hennequin G, et al. Cortical-like dynamics in recurrent circuits optimized for sampling-based probabilistic inference.
• Qi Y, Gong P. Fractional neural sampling as a theory of spatiotemporal probabilistic computations in neural circuits.
• Zhang W H, Wu S, Josić K, et al. Recurrent circuit based neural population codes for stimulus representation and inference.
• Chu T, Ji Z, Zuo J, et al. Oscillatory Tracking in Continuous Attractor Neural Networks Accounts for Theta Phase Shift of Hippocampal Place Cells.
• Zhang W, Wu Y, Wu S. Equivariant Representation in Recurrent Networks with a Continuous Manifold of Attractors.

[NorbertZheng]

Overview

Neural coding, or how the brain encodes information, is a scientific problem at the "crown" level in computational neuroscience. Its importance is almost self-evident, as we seek to fundamentally elucidate the higher cognitive functions of the brain and inevitably answer how information is encoded by neural activity. But unfortunately, or confusingly (see more discussion later), we still don't have a clear answer to this question so far.

Broadly speaking, any neural computational modeling involves more or less encoding of information, i.e., specifying the form of input information in the model. However, the definition of neural coding (Fig. 1A) in this paper is narrower, which specifically refers to

• how the spiking activity generated by the neuron group after receiving the external input encodes the input information (autoencoding?).

Our group has recently completed several works on neural coding, which prompted me to write this somewhat reminiscent academic essay for three reasons:

• I have done three studies on the neural coding problem before and after, in 2000, 2010, and 2022. The time span is more than 20 years. After each completion, I completely put down the work at that time and switched to doing it. Other problems were not expected to be picked up again after more than 10 years. Personally, I feel that it is a very interesting academic experience, and I feel a little "impermanent" on the academic road, so I wrote it down and shared it with everyone, especially young scholars.
• Neural coding is a particularly interesting problem in itself, and while it's very mathematical in nature, there are often philosophical questions behind it (which is why I love brain science). Many times I don't understand it, so I write it down for everyone to think about.
• Brain-inspired intelligence is getting more and more attention now and is considered to be an important way for the development of artificial intelligence, but so far, it seems that few people have seriously thought about how to apply neural coding strategies to AI (the application I refer to here). Not just simply using the concept of spiking, but really drawing on the specific mechanisms by which the neural system encodes information). Therefore, I wrote this essay in the hope that it will also be beneficial to AI workers.

Figure 1: Neural encoding background knowledge. A. External stimuli are encoded as the activity of neuron groups in the brain, which mediate our perception and cognitive behavior; B. The shape of the pulses emitted by neurons is almost unchanged, and the information is contained in the pulse sequence generated by neurons; C. A continuous variable is encoded by a group of neurons whose tuning functions overlap and cover the parameter space; the neural network underpinning this group encoding is a continuous attractor network; D. Under constant input, the neuron firing pulse trains are highly independent Rules, the results of each experiment record are different.

[NorbertZheng]

Background

Spike firing of neurons

The brain perceives the external world through different senses (vision, hearing, smell, etc.), and these physical or chemical signals (light, sound waves, odor molecules, etc.) that contain information about the external world are uniformly converted into the electrical activity of neurons within the brain , that is, the neuron's sequence of firings (spike train). Each discharge is called a spike or action potential (Figure 1B). The shape of the spikes is nearly constant, so from an information-encoding perspective,

• it is the neuron-generated spike train rather than the spike shape that contains the input information.

[NorbertZheng]

Single neuron coding

Early research in the field focused on how a single neuron encodes information. There may be two extreme coding methods for the pulse sequence based on the response of a single neuron:

• rate coding: Information is encoded in the firing frequency of the neuron, that is, the number of pulses per unit time. The specific temporal structure is irrelevant.
• temporal coding: Information is encoded in the fine structure of the neuron's spike train. This coding strategy is employed for nighttime localization in the owl's auditory system, but is difficult to achieve in the cerebral cortex, where is so full of noise that the fine temporal structure of the pulse train is difficult to maintain.

In addition, there are some coding hypotheses between rate coding and temporal coding, which mainly

• consider some high-order statistical properties of neuron spike trains in the time domain.

In general, it is difficult to answer the information encoding principles of the brain at the single neuron level, because the brain always has a large number of neurons involved in processing information, and these neurons interact through synaptic connections, isolated single neurons computation does not exist. Therefore, to decipher the laws of neural coding, we also need to start with the laws of neuron group activity (see below).

[NorbertZheng]

Neuron population coding

In the representation of some continuous variables, such as object orientation, head orientation, movement direction, spatial location, etc., experiments have confirmed that

• the brain encodes information through the coordinated activity of a group of neurons.

Taking the head orientation as an example, its value ranges from 0 to $2\pi$, and the neural system covers the entire variable value space with the tuning function of a large number of neurons (that is, the range of stimuli values ​​that neurons respond to): each neuron is mainly "responsible for" an angle value (preferred stimulus), that is, when the external stimulus takes this value, the neuron responds most strongly; at the same time, the tuning functions of a large number of neurons overlap each other (Fig. 1C). This way, when encoding a particular external stimulus value, a group of neurons, rather than a single one, responds, so that noise can be averaged out and the stimulus value can be decoded more accurately. The neural network structure that supports this group encoding is the Continuous Attractor Neural Network (CANN) (it is not the focus of this article and will not be described in detail here. Interested readers can refer to the WeChat public account article: Continuous Attractor Network). The neural system's group coding strategy has been supported by a large number of experiments, but

• the current experimental data only statistically indicate that the neural system encodes with a large number of neurons,

and does not answer many details of the encoding process (see below).

[NorbertZheng]

Maybe disentangling representation has better explanation about why single neuron responds to single task factor.

[NorbertZheng]

Irregular firing of neurons

One of the most puzzling phenomena in neural coding is the irregular firing of neurons under constant input: experiments have found that even with the same input, each neuron fires a different sequence of spikes (Fig. High degree of irregularity (irregular), the degree of clutter even exceeds that of Poisson random processes.

• This random firing is only present in living networks, isolated single neurons firing regularly under constant input, so this is a network effect.

The neural network structure underpinning this random firing is the excitation-inhibitory balance network, which has been supported by numerous neurobiological experiments and computational modeling. But what is particularly puzzling is why the brain uses this random neural activity to encode information? Intuitively, isn't the simplest and most reliable encoding a neuron responding constantly to constant input, as we do with encoding strategies in communication networks? Answering the computational mechanics behind the erratic activity of neurons could open the way for us to decipher the mysteries of neural coding.

[NorbertZheng]

In-context learning may provide better explanation about this issue.

[NorbertZheng]

Three research experiences on neural coding

Below is a description of the three research work I have done in neural coding over the past 20+ years. In order to describe clearly, it is necessary to introduce some mathematical notation first. $x$ is the value of a continuous variable, which can be the orientation of the object, the direction of movement, the spatial position, etc. Denote $f(c,x)$ as the tuning function of the neuron with the stimulus preference value $c$, that is, the average response curve. Under a single observation, the firing frequency of a single neuron can be written as

$$ r(c)=f(c,x)+\epsilon, $$

where $\epsilon$ is the noise. It is expressed by a probability model, that is, when a neuron $c$ is given a stimulus value $x$, the probability of the firing frequency $r$ is $p(r|x)$. The specific form of the conditional probability $p(r|x)$ here is determined by the characteristics of the noise. For example, $\epsilon$ is Gaussian noise with $0$ mean and $\delta^{2}$ variance,

$$ p(r|x)=\frac{1}{(\sqrt{2\pi}\delta)\exp\left(-\frac{[r-f(c,x)]^{2}}{2\delta^{2}}\right)}, $$

We use vectors $\mathbf{r}$ to represent the activity of neural populations. Mathematically, given a conditional probability $p(\mathbf{r}|x)$ (also called a likelihood function, generative model, etc.), it is equivalent to clarifying the coding process of the nervous system.

[NorbertZheng]

Unreliable Decoding, 2000 at the Institute of Physics and Chemistry, Japan

This is the first neural coding job I've done. Around 2000, a number of international research groups simultaneously theoretically explored the mathematical properties of neuron group coding, mainly based on three mathematical methods:

• Directly compute Fisher information of $p(\mathbf{r}|x)$. According to Cramer-Rao bound, the reciprocal of Fisher information gives the lower bound of the error of any unbiased decoding method, so Fisher information measures the amount of stimulus information contained in neuron groups in a sense.
• The simplest decoding method, Population Vector or Center of Mass, is used, which is equivalent to a weighted average of the preferred stimulus of all neurons according to the neuron's response strength. This method is the most intuitive, but the disadvantage is that the error is too large.
• Use the maximum likelihood method $\hat{x}=Max_{x}lnp(\mathbf{r}|x)$, which is statistically optimal, the Cramer-Rao bound can be approximated asymptotically (sufficient number of neurons) when the correlation between neuronal activity is small enough.

[NorbertZheng]

I was just getting into computational neuroscience at the time, and my knowledge of neuroscience was very small. Theoretical analysis of neuron group coding was one of the few topics that I could get started with right away. I remember one weekend night, while thinking about how to do something new for a long time, I suddenly realized that

• the maximum likelihood method or Fisher information calculation has an implicit assumption that the likelihood function $p(\mathbf{r}|x)$ is known by the decoder.

But in real scenarios, it is unknown to the neural system, especially the statistical properties of noise, because noise can come from many aspects, including the external environment and input from other brain areas, which are beyond the control of the brain.

Therefore, neural coding actually implies a very interesting mathematical problem, that is, the brain needs to decode the information of the external input under the condition that only part of the coding process is known. I call this situation unreliable decoding (unfaithful decoding), without loss of generality, its mathematical form can be written as:

$$ \hat{x}=Max_{x}lnq(\mathbf{r}|x), $$

where $q(\mathbf{r}|x)$ represents an encoding model set by the decoder, which may not be equivalent to the real encoding process $p(\mathbf{r}|x)$. The accuracy of this decoding method is solvable under certain conditions (see [1, 2] for details).

[NorbertZheng]

My collaborators and I analyzed the performance of neuron group encoding when the formation of the likelihood function $p(\mathbf{r}|x)$ was different (Figure 2), especially the performance of neuron group coding when $p(\mathbf{r}|x)$ ignores the correlation of neuron activity, and the theory proved the corresponding decoding method in this case. It is pattern matching (template matching), and pattern matching can be realized naturally by biologically plausible continuous attractor network [2]. The work was submitted to the NIPS of the year (now called NeurIPS; it took only two weeks from the idea to the submission, and the NIPS emphasized the novelty of the idea rather than the completeness of the work, which is different from today's requirements), and the result is Best score of all my NIPS articles (score: 5, 5, 4; confidence is 5, 5, 5; 5 is the highest score at the time). After completing this work, I then did a group coding work, which considered how the previous decoding value can be used as a priori knowledge to continuously optimize the subsequent decoding accuracy during the decoding process iteratively over time, which is equivalent to a time dimension. Bayesian inference on [3]. Since then, my work interest has shifted to Continuous Attractor Networks (CANN), and the research on neural decoding has been around for nearly ten years.

Figure 2: Unreliable neural decoding. A. Mathematical expression of unreliable decoding, and precision calculation formula; B. Under the condition of ignoring the correlation of neuron activity, unreliable decoding realizes pattern matching, and its precision and computational complexity are both in the maximum likelihood method and Between the centroid methods, a balance between accuracy and computational complexity is achieved.

[NorbertZheng]

Dynamic Coding, 2010, Institute of Neurology, Chinese Academy of Sciences

My renewed interest in neural coding was around 2010. At that time, an article by a Japanese friend quoted our unreliable decoding work, and used the mathematical formula we introduced to measure the stimulus information contained in the neuronal association activity [4], an application that I had not thought of before. At that time, I happened to be collaborating with Professor Liang Peiji from Shanghai Jiaotong University. Now that we were familiar with this mathematical analysis tool, we thought of using this tool to explore an important scientific problem, namely, the coding strategy of the nervous system during the adaptation process. Adaptation is a universal behavior of the nervous system that occurs everywhere in life. Judging from the response of a single neuron, when a stimulus just appears in the receptive field of the neuron, the neuron will have a strong response, but slowly the neuron "adapts" to the stimulus, which is reflected in the response intensity returning to the stimulus. The background level before appearing. According to the traditional view of frequency encoding, this means that neurons no longer encode stimulus information, but this seems to be inconsistent with common sense in our lives. For example, for an object that appears in the field of view for a long time, the visual system does gradually "adapt" to it, but we still know that the object is still there, just no longer pay attention to it deliberately. So

• how is the information of the object continuously encoded after adaptation?

To answer this question, we propose a dynamic coding hypothesis (Fig. 3), that is, during the adaptation process of neurons, the nervous system does not completely ignore the external stimulus information, but the coding strategy changes, that is, from the initial Fast frequency coding translates into energy-efficient low-frequency correlation code (ie, the nervous system encodes stimulus information with low-frequency synchronous activity of neuronal populations). The advantage of this dynamic encoding is that when a new stimulus is first presented, the nervous system can respond quickly, which is most critical to the survival of the animal; but as the stimulus is recognized, to conserve energy, the nervous system switches to More economical associative coding to maintain information. In order to test this hypothesis, Liang Peiji's research group recorded the adaptation process of the bullfrog retina to dark stimuli (the bullfrog was chosen because its adaptive response was as long as 5 seconds, which provided us with good experimental data), and analyzed the firing frequency of neurons The results support our hypothesis by the transformation of the amount of stimulus information contained in the correlation between early and late adaptation.

When this work was submitted, the reviewers all said they liked the idea of dynamic coding, but they also doubted that the amount of stimulus information we calculated was too small (only a dozen neurons could be effectively recorded in the experiment at that time), and it was not ruled out that it was the data Analyzing the possibility of errors requires us to analyze the data with simpler and more intuitive methods (classifier, synergy information). After some tossing, the work was finally published in the Journal of Neurophysiology [5]. But I have always believed that dynamic coding strategies are widespread in the nervous system, because this is very natural and effective, but more experimental data are needed to prove it. In addition, what I am satisfied with this work is that the theoretical hypothesis is in the front, and the experimental design and verification are in the back. After completing this work, my collaborators and I have done some extended work piecemeal. For more details, please refer to the WeChat work account article: [Dynamic Neural Coding](). Since then, my research on neural coding has been on hiatus for almost a decade.

Figure 3: Dynamic Neural Coding. A. Typical neuronal adaptive behavior, the neuronal firing frequency gradually decreases during the adaptation process; B. During the adaptation process of the bullfrog retina, the resolution of Meta-fire rate classifiers are getting lower and lower resolution; C. Schematic illustration of dynamic coding work. When the external stimuli first appeared, the bullfrog retina transmits information rapidly to subsequent neurons through the high-frequency independent firing of neurons; after adapting to the external stimulus, the bullfrog retina transmits information backward through the associated low-frequency firing between neurons.

[NorbertZheng]

Random sampling coding, Peking University in 2022

After completing the above two tasks, my research on neural coding stopped for a long time, and my interest was rekindled due to the influence of Dr. Zhang Wenhao, a graduate student, who invited me to discuss the irregular firing of neurons a few years ago. computational meaning. The following highlights a work we accepted at NeurIPS this year [6].

As mentioned earlier, one of the most puzzling phenomena of neural coding is the irregular firing of neurons under constant input. There are several hypotheses about the computational significance of this random distribution. The hypothesis I personally agree with the most is random sampling coding, which holds that the random firing of neuron groups is actually doing sampling-based Bayesian inference [7]. A large number of human behavior experiments and animal physiological experiments support that the brain can do Bayesian inference with statistical optimization, but to perform Bayesian inference specifically, the nervous system needs to calculate the posterior distribution of information. It is unclear whether, or whether, neural circuits represent a probability distribution.

• The random sampling point of view believes that the random firing activity of neurons is to acquire knowledge of the posterior distribution by sampling (just like diffusion-model, which provides a sampling method to get more samples to approximate the true distribution, but diffusion-model never masters the whole distribution), rather than to directly represent the posterior distribution, so that we can perceive the uncertainty of the external world.

This kind of uncertainty knowledge is very important for brain cognition. For example, when recognizing objects, we must not only identify the category of the object, but also have a judgment on the reliability of the recognition result.

In the field of machine learning, approximating a complex probability distribution with random sampling is a classic problem. The traditional random sampling algorithm is MCMC (Markov Chain Monte Carlo), but this algorithm is very slow. To this end, the field of machine learning has developed an improved and sophisticated algorithm called the Hamilton Monte Carlo (HMC) method. Its basic idea is to introduce a new momentum variable (that is, the speed of the sampled variable), which together with the sampled variable constitutes a physical Hamiltonian dynamic system; through relatively simple random sampling of momentum, indirect The random change of the sampled amount is realized, thereby greatly accelerating the whole sampling process. Years of research experience in intersecting fields tell us that usually a clever algorithm of artificial intelligence is often applied to biological systems, and vice versa. Indeed, many international research groups have studied the mechanism of HMC in the nervous system, such as [8, 9]. We are also interested in this question. In the published work of NeurIPS [6], we use the familiar continuous attractor neural network with negative feedback to rigorously prove theoretically that in the presence of noise, the network can implement HMC to sample the posterior of a continuous variable distribution (Figure 4). The mechanism behind it can be intuitively understood as follows:

• From the dynamic effect, the negative feedback increases the mobility of the network attractor state.
• From the sampling effect, the negative feedback plays the role of momentum in the HMC
• Since the negative feedback depends on the activity of the network Therefore, it effectively speeds up the random sampling of network dynamics, which is similar to the introduction of a momentum term in the gradient descent algorithm of machine learning.

For more details, please refer to our follow-up WeChat public account article (if Dong Xingsi, the first author of the article, is willing to write).

Figure 4: A neural network implements random sampling-based neural coding. A. An example of Bayesian inference; B. A continuous attractor neural network model with negative feedback; C. An example of the network sampling in the state space; D. The network implements HMC, which greatly speeds up the sampling process of the posterior distribution.

[NorbertZheng]

Some thoughts on neural coding

The above describes the three research experiences that my collaborators and I have conducted on neural coding in the past 20 years. You can also see that the specific mechanism of neural coding in the field is still very unclear so far, and the research progress has been slow for so many years. This also prompted me to reflect on the scientific problem of neural coding itself, which I would like to share with you below.

[NorbertZheng]

Is there a uniform law for neural coding?

From the perspective of a theoretical researcher, I certainly hope that there is a unified law, just as Newton's three laws of physics describe the laws of motion of nature at low speeds, but there may not be such a universal law in brain science. Sexual coding rules. The golden rule of life science is Darwin's theory of evolution. The development of the brain is more like an engineering problem. It is a long-term evolution of living organisms to adapt to the environment, not pre-designed. Different species face different computing requirements, and different organs of the brain are also processing different information. Therefore, different species or different brain regions of the same species may adopt different information encoding strategies. There is no reason to assume that the nervous system must use the same information. Laws encode information. When we study neural coding, we need to be careful to define the scope of the law to apply.

Of course, there is also a possibility, because life on earth faces a similar natural environment, neurons all use the signal representation and transmission method of pulse firing (possibly from energy saving and other needs), and the nervous system also uses the network for calculation. etc., it is not excluded that

• under the constraints of these common conditions, the neural coding strategy has evolved to a mathematically unique optimal solution, so it is unified.

[NorbertZheng]

What is the neural code encoding?

In the traditional animal experimental paradigm (and corresponding mathematical modeling), we present the experimental animal with a stimulus variable that we "define" in advance, such as a moving grating to define an orientation variable; then we record and analyze the neuron population. activity, in the hope of deciphering how this population of neurons encodes the orientation values ​​represented by the grating. Note that the entire research paradigm implicitly assumes a strong assumption that experimental animals are encoding an orientation variable as we wish. But this hypothesis is unprovable because animals can't speak. For the experimental animal, all it sees is a moving grating. It is very likely that the response of neurons is not encoding the orientation variable (orientation itself is actually a very abstract mathematical concept), but encoding its life in the image. A piece of information that makes sense for an activity. This is somewhat similar to the problem faced by deep learning networks in black-box training: if we always use images of horses and grass to train horse recognition, the network will most likely recognize grass as horses. Deep learning networks can try to overcome this by increasing the diversity of samples, but in animal experiments, adding too many stimulus samples is not feasible due to experimental costs (and theoretically we don't know exactly how many samples are appropriate) .

[NorbertZheng]

When introducing random sampling coding above, it was also mentioned that the brain not only encodes the properties of objects, but also encodes other related information, such as the uncertainty of objects. For example, for a fuzzy animal image, we may recognize that it is a horse with a 60% probability, and recognize that it is not a horse but a donkey with a 40% probability. The calculation of this uncertainty requires comparison with prior knowledge (ground truth), which must be pre-stored in the brain rather than temporarily provided by external input. As seen from this example, neural coding may involve complex cognitive processes, not just direct responses to external inputs. This is also reflected in the experimental definition of neuronal receptive fields. The receptive fields of neurons in the retina, LGN, and V1 are relatively easy to determine, but the receptive fields of neurons in the V2 and above brain regions have not been determined, because the neurons in these brain regions have received a large number of feedback connections from other brain regions. Influence, their responses are no longer simply driven by external input, so the traditional definition of receptive field no longer holds.

[NorbertZheng]

There is a logical debate in Chinese history that "a white horse is not a horse". Applied to neural coding, when recognizing a picture of a horse, does the brain encode an abstract concept (prototype) of all horses, or does it encode the specific (white) horse that is presented in the picture? I prefer the former, because life experience tells me that we can easily identify horses, but it is difficult to describe what horses look like when we close our eyes; at the same time, from an evolutionary perspective, we do not need to encode the specific appearance of all horses. Related to this concept, Zhang Wenhao et al. did an interesting neural coding work [10]. Different from the traditional only considering $p(\mathbf{r}|x)$ as the encoding process, we consider the variable $x$ itself is generated by a latent variable $z$, ie. $p(x|z)$. Here, $x$ corresponds to a specific image of a horse that generates a neural response $\mathbf{r}$, and the hidden variable $z$ corresponds to the prototype of the horse, which is the information that the brain really needs to encode. This new coding framework can explain some experimental findings that traditional models cannot, including that the widespread lateral connections between neurons can be explained as a store of prior knowledge $p(x|z)$ (e.g. in-weight learning?).

[NorbertZheng]

Neural coding is a false proposition?

The concept of coding, and its corresponding mathematical description - information theory, originally came from the field of communication, which considers how to encode information into a suitable signal and transmit it safely to a distant place, and then decode it back, so the research of coding is actually signal transmission instead of information processing. Thinking about it carefully, there is no simple signal transmission task in the brain. The transmission of neural activity from one brain area to another involves the extraction and processing of information; in the brain, the transmission of neural signals and information processing are intertwined. together. Therefore, research on neural coding based on signal transmission is incomplete and may even mislead our understanding of the computational function of the brain. For example, in current neural coding modeling work, we usually start by assuming an encoding process $p(r|x)$, and then analyze how neural activity characterizes and decodes this input signal $x$; but the reality may be that the brain directly computes and characterizes another variable $y(x)$.

• The framework of information theory is not unusable, but we need to understand its limitations in describing the information processing mechanism of the brain.

[NorbertZheng]

Can neural coding be deciphered?

From the above discussion, the strategy of neural coding may not be unique, and animal experiments cannot really determine the content of neural coding. Neural coding and computation are intertwined, which seems to mean that it is difficult for us to decipher the mystery of neural coding. So is it finally possible for us to decipher the mechanisms of neural coding? I don't have an answer either, but I realize that neural coding faces the same dilemma as deep learning interpretability. As we all know, after setting a specific task (usually a classification task) and training a deep learning network with big data, it is very difficult to decipher the computing mechanism inside the network. The same is true for neural coding research: the brain adapts to its environment (task) and evolves (trains) to process information more efficiently, but it is extremely difficult to decipher how neural circuits specifically encode information.

• Because the structure and computational laws of brain networks are task-driven like deep learning networks, not after designing the coding strategy of neurons.

This can also be reflected from the working principle of the library network (reservoir network):

• the external input information is projected into a high-dimensional neural activity representation space, which is convenient for the next-level neurons to read, as for how the input information is encoded, it is difficult to express with simple rules.

Therefore, I tend to think that for some simple inputs, it is possible to describe how they are encoded in the early stages of neural pathways; but for more complex inputs, and their computations in advanced brain areas process, it is difficult to quantitatively describe it with neural coding. Perhaps, like deep learning, we can often only describe the working principle of a neural system as a whole, the so-called black-box operation.