Do Language Models Have a Global Workspace?
3 days ago
Sunday, September 30, 2007
Might stimulate some thinking (from the same paper)
Inversion of expression. Specific gene-expression profiles map onto different discharge response types. The most striking example is a near-perfect inversion of the expression profile between cells that discharge initially with a burst onset (b-subtype) and those with a delayed onset (d-subtype)85. So, even cells that are normally classified in the same broad class, such as fast spiking, can have diametrically opposite expression profiles depending on the onset response. This finding also indicates that only a few transcription factors might control the expression of entire sets of ion-channel genes, in which case it is probable that different combinations of transcription factors would give rise to a finite number of distinct electrical classes.
Neuro-circuits & Interneurons
Interestingly, the static (quantal) and dynamic (depression and facilitation) properties of facilitating synapses from single pyramidal neurons onto interneurons vary across layers126, which might cause targeted inhibitory cells in deep cortical layers to discharge before those in supragranular layers. Such layer-specific differences in the recruitment of interneurons could influence the direction of information flow in the cortical column.
Remarkably, all the synapses formed by one interneuron onto multiple pyramidal neurons show identical synaptic dynamics15. All the synapses from an interneuron onto all targets of the same type (pyramidal neurons in this case) seem to have identical release probabilities and time constants for recovery from synaptic depression and facilitation. This homogeneity principle contrasts sharply with the heterogeneity of glutamatergic synapses formed by a pyramidal neuron onto other pyramidal neurons and also has implications for the forms of learning that might shape these synapses. The absolute strength of these synaptic connections is heterogeneous (probably due to different numbers of synapses and/or postsynaptic receptors), indicating that they could be modified by the relative timing of activity in only one pair of neurons (presynaptic and postsynaptic), but their dynamics are homogeneous, suggesting that these parameters must be modified by the activity patterns of the entire population of postsynaptic pyramidal neurons relative to the single presynaptic interneuron.
Why balance Yang with Yin? Why does excitation need to be balanced with inhibition and why do transient moments of imbalance occur? This is a vast area, which will not be dealt with in this review, except to speculate on two potential reasons. At the level of individual neurons, matching inhibition as a function of stimulus intensity could allow information to be processed and encoded at a higher or lower temporal resolution, depending on the baseline firing rates. This can be achieved by changing the membrane time constant, which changes the time window for temporal integration161 (see also Ref. 162) and by changing the temporal precision of spike generation by adding high-frequency membrane 'noise'163, 164, 165. At this level, balance might be required to normalize the baseline for synaptic integration as a function of activity (to normalize the mutual information between the input channels) and spiking might reflect moments of imbalance (high mutual information between the input channels). At the microcircuit level, a sliding scale between integration and coincidence detection as a function of activity in each neuron could be important to control which neurons synchronize at which frequencies162. Balance might be required to keep all neurons independent (to normalize mutual information across neurons) and oscillations might reflect orchestrated momentary imbalances of groups of neurons (high mutual information between neurons). Needless to say, considerable work is required to test and turn theory into fact.
http://www.nature.com.ezproxy.hsclib.sunysb.edu/nrn/journal/v5/n10/full/nrn1519.html
Remarkably, all the synapses formed by one interneuron onto multiple pyramidal neurons show identical synaptic dynamics15. All the synapses from an interneuron onto all targets of the same type (pyramidal neurons in this case) seem to have identical release probabilities and time constants for recovery from synaptic depression and facilitation. This homogeneity principle contrasts sharply with the heterogeneity of glutamatergic synapses formed by a pyramidal neuron onto other pyramidal neurons and also has implications for the forms of learning that might shape these synapses. The absolute strength of these synaptic connections is heterogeneous (probably due to different numbers of synapses and/or postsynaptic receptors), indicating that they could be modified by the relative timing of activity in only one pair of neurons (presynaptic and postsynaptic), but their dynamics are homogeneous, suggesting that these parameters must be modified by the activity patterns of the entire population of postsynaptic pyramidal neurons relative to the single presynaptic interneuron.
Why balance Yang with Yin? Why does excitation need to be balanced with inhibition and why do transient moments of imbalance occur? This is a vast area, which will not be dealt with in this review, except to speculate on two potential reasons. At the level of individual neurons, matching inhibition as a function of stimulus intensity could allow information to be processed and encoded at a higher or lower temporal resolution, depending on the baseline firing rates. This can be achieved by changing the membrane time constant, which changes the time window for temporal integration161 (see also Ref. 162) and by changing the temporal precision of spike generation by adding high-frequency membrane 'noise'163, 164, 165. At this level, balance might be required to normalize the baseline for synaptic integration as a function of activity (to normalize the mutual information between the input channels) and spiking might reflect moments of imbalance (high mutual information between the input channels). At the microcircuit level, a sliding scale between integration and coincidence detection as a function of activity in each neuron could be important to control which neurons synchronize at which frequencies162. Balance might be required to keep all neurons independent (to normalize mutual information across neurons) and oscillations might reflect orchestrated momentary imbalances of groups of neurons (high mutual information between neurons). Needless to say, considerable work is required to test and turn theory into fact.
http://www.nature.com.ezproxy.hsclib.sunysb.edu/nrn/journal/v5/n10/full/nrn1519.html
Kv3 channels
Analysis of the Kv3 subfamily of K+ channel subunits has lead to the discovery of a new class of neuronal voltage-gated K+ channels characterized by positively shifted voltage dependencies and very fast deactivation rates. These properties are adaptations that allow these channels to produce currents that can specifically enable fast repolarization of action potentials without compromising spike initiation or height. The short spike duration and the rapid deactivation of the Kv3 currents after spike repolarization maximize the quick recovery of resting conditions after an action potential.
The ability to fire action potentials at high frequencies (often up to 1 kHz) or to follow high-frequency inputs are important physiological functions of numerous cells throughout the mammalian CNS. Neuronal populations within such disparate regions as the hippocampus, basal ganglia, neocortex, reticular thalamus, medial vestibular nucleus and auditory nuclei are capable of responding to afferent input with action potentials of brief duration and of firing repetitively at high frequencies.
Although numerous channel types have been implicated in conferring such properties, the voltage-gated K+ channels of the Kv3 subfamily have now been identified as major determinants of the FS phenotype.
Molecular characteristics of Kv3 subfamily members
Both rodents and humans possess four Kv3 genes: Kv3.1, Kv3.2, Kv3.3 and Kv3.4 ([6 and 7]). All four Kv3 genes generate multiple protein isoforms by alternative splicing, which produces versions with different intracellular C-terminal sequences. There are now 13 different Kv3 proteins known in mammals (Kv3.1a–Kv3.1b, Kv3.2a–Kv3.2d, Kv3.3a–Kv3.3d and Kv3.4a–Kv3.4c), yet the currents expressed in heterologous expression systems by the spliced isoforms of each Kv3 gene are virtually indistinguishable. Recent studies suggest that the alternatively spliced C termini confer isoform-specific regulation by second messenger signaling systems and targeting to distinct neuronal compartments.
The ability to fire action potentials at high frequencies (often up to 1 kHz) or to follow high-frequency inputs are important physiological functions of numerous cells throughout the mammalian CNS. Neuronal populations within such disparate regions as the hippocampus, basal ganglia, neocortex, reticular thalamus, medial vestibular nucleus and auditory nuclei are capable of responding to afferent input with action potentials of brief duration and of firing repetitively at high frequencies.
Although numerous channel types have been implicated in conferring such properties, the voltage-gated K+ channels of the Kv3 subfamily have now been identified as major determinants of the FS phenotype.
Molecular characteristics of Kv3 subfamily members
Both rodents and humans possess four Kv3 genes: Kv3.1, Kv3.2, Kv3.3 and Kv3.4 ([6 and 7]). All four Kv3 genes generate multiple protein isoforms by alternative splicing, which produces versions with different intracellular C-terminal sequences. There are now 13 different Kv3 proteins known in mammals (Kv3.1a–Kv3.1b, Kv3.2a–Kv3.2d, Kv3.3a–Kv3.3d and Kv3.4a–Kv3.4c), yet the currents expressed in heterologous expression systems by the spliced isoforms of each Kv3 gene are virtually indistinguishable. Recent studies suggest that the alternatively spliced C termini confer isoform-specific regulation by second messenger signaling systems and targeting to distinct neuronal compartments.
Nature of Random, Random of nature
What's nature? The property innate in it, is something called radom.
Through infinite length of time, it created the ball like thing, stars. It create a network of these stars. And all the things it made up make up itself, nature. All this things take place like a miracle, but you can still imagine them happen.
But there's something hard to believe, is these random nature can create living things. It has no powful tools, except for using two things: randomness, and time. And today, when we, human beings, the highest achievement of nature, with our somewhat intelligant brains, coming to try to unravel the mechanisms produced by random, still totally get lost. Assembly of random happenings, v.s. those intelligent scientific brains, the biggest joke hah? But still, the latter still striving hard.
Nature depends on time to evolve. But we need to compete against time.
Through infinite length of time, it created the ball like thing, stars. It create a network of these stars. And all the things it made up make up itself, nature. All this things take place like a miracle, but you can still imagine them happen.
But there's something hard to believe, is these random nature can create living things. It has no powful tools, except for using two things: randomness, and time. And today, when we, human beings, the highest achievement of nature, with our somewhat intelligant brains, coming to try to unravel the mechanisms produced by random, still totally get lost. Assembly of random happenings, v.s. those intelligent scientific brains, the biggest joke hah? But still, the latter still striving hard.
Nature depends on time to evolve. But we need to compete against time.
Thursday, September 27, 2007
Meeting Buzsaki
So, how does everything goes with Gyorgy Buzsaki?
Well, mixed feelings.
This is one of the greatest scientific talk that I have ever experienced. The whole conference room is silence, entire silence. The talk is interesting, not because of showing some fantastic, funny while amazing things, but by the way it's clearly shown. The speaker just draw you into the deep world of his logical world. You can't deny anything because everything is so convincing, and because you just want to get into the new insight he opened to you try to imagine further. I'm just wondering what he can do in his next talk if he want to use different slides, because this series of slide is almost perfect. Perfectly clear and reasonable.
You might imagine the question section to be even hotter than last time's. Actually no. Paul Adams is also there, but he didn't question anything. Unusualy, this time's faculty tend to be silent, while students asked some questions. You know in normal times, the faculties came up with a lot of brilliant questions. But this time, few of them asked, and actually the ones even I can come up with an answer.
David say he seems not welcoming questions. Well, think over, that's seems true. Most of the seminar speakers I see, will specifically encourage people to ask and don't forget to give feedback to the questions they asked. He didn't do that specially. I'm sure he likes people asking questions. But the lecture he made, seems to give no chance for people to ask questions. I still remember when we went to talk with him about some questions after the talk, he would tell us that there's foible where where on his own.
Actually I didn't ask any questions except for asking if there are any way to get an discount on his book. :) Everyone got into laugh including him. But he did answer my questions seriously. Normally I asked a lot of questions during the one and a half hour discussion, but you know what, this time is mainly he asking us questions! He asked us why choose neuroscience rather any other scientific subjects, why neuroscience rather that other biology branches; what's the difference between people in biotechnology and us. He said I'm sure none of you come to this field aim to publish a paper. Silence this time, because everyone can't imagine how you can get graduated without publishments, how we can get our future jobs without a paper. Then he said do any one of you worry about what kind of position you'll get later, we don't care, right? Silence again, of course you don't need to worry about anything like this. But you are true, when we just choose this field, we don't think about papers, tenures -- only science.
All the questions he asked I never gave out the one he wanted. He gave an example that imagine there's 10billion treasure underneath somewhere of SB campus, what's the quickest way to get it. I said buy the whole campus then you can do whatever you want. Of course I mistook him, he is looking for a scientific answer. The securest way is to dig the campus square by square, but the quickest way is just random search. All through the whole lunch time is like this, he raised many questions, we are like headless flies, try to think and answer as much as possible but never got his clue.
Well, mixed feelings.
This is one of the greatest scientific talk that I have ever experienced. The whole conference room is silence, entire silence. The talk is interesting, not because of showing some fantastic, funny while amazing things, but by the way it's clearly shown. The speaker just draw you into the deep world of his logical world. You can't deny anything because everything is so convincing, and because you just want to get into the new insight he opened to you try to imagine further. I'm just wondering what he can do in his next talk if he want to use different slides, because this series of slide is almost perfect. Perfectly clear and reasonable.
You might imagine the question section to be even hotter than last time's. Actually no. Paul Adams is also there, but he didn't question anything. Unusualy, this time's faculty tend to be silent, while students asked some questions. You know in normal times, the faculties came up with a lot of brilliant questions. But this time, few of them asked, and actually the ones even I can come up with an answer.
David say he seems not welcoming questions. Well, think over, that's seems true. Most of the seminar speakers I see, will specifically encourage people to ask and don't forget to give feedback to the questions they asked. He didn't do that specially. I'm sure he likes people asking questions. But the lecture he made, seems to give no chance for people to ask questions. I still remember when we went to talk with him about some questions after the talk, he would tell us that there's foible where where on his own.
Actually I didn't ask any questions except for asking if there are any way to get an discount on his book. :) Everyone got into laugh including him. But he did answer my questions seriously. Normally I asked a lot of questions during the one and a half hour discussion, but you know what, this time is mainly he asking us questions! He asked us why choose neuroscience rather any other scientific subjects, why neuroscience rather that other biology branches; what's the difference between people in biotechnology and us. He said I'm sure none of you come to this field aim to publish a paper. Silence this time, because everyone can't imagine how you can get graduated without publishments, how we can get our future jobs without a paper. Then he said do any one of you worry about what kind of position you'll get later, we don't care, right? Silence again, of course you don't need to worry about anything like this. But you are true, when we just choose this field, we don't think about papers, tenures -- only science.
All the questions he asked I never gave out the one he wanted. He gave an example that imagine there's 10billion treasure underneath somewhere of SB campus, what's the quickest way to get it. I said buy the whole campus then you can do whatever you want. Of course I mistook him, he is looking for a scientific answer. The securest way is to dig the campus square by square, but the quickest way is just random search. All through the whole lunch time is like this, he raised many questions, we are like headless flies, try to think and answer as much as possible but never got his clue.
Wednesday, September 26, 2007
The heart v.s. the brain
One question I want to ask Gyorgy Buzsaki:
When we look at the wiring mechanism of neurons, is the heart comparable to the brain in some way?
Both in the heart and the brain, there're electrical events going on, there's resting potential and depolarization, there're similarly functioning ion channels. Oh, gap junctions, that's special in the heart. One difference is that, the target of the electrical signals are actually contented inside the heart, i.e., the whole events from stimulation to response, happens in the same organ. In contrast, it's more likely that the brain gives signal, and the outside receptors response.
But when we consider the electrical wiring of the brain, is it possible to use the heart as a simple model? You know, many ion channelists favor to choose the heart as the target organ for research, which contain simliarly functioning ion channels, but much easier for them to unravel the mechanisms underlying.
Or, given the gap junction thing, no way?
Well, I need to think more before I went up to ask him.
When we look at the wiring mechanism of neurons, is the heart comparable to the brain in some way?
Both in the heart and the brain, there're electrical events going on, there's resting potential and depolarization, there're similarly functioning ion channels. Oh, gap junctions, that's special in the heart. One difference is that, the target of the electrical signals are actually contented inside the heart, i.e., the whole events from stimulation to response, happens in the same organ. In contrast, it's more likely that the brain gives signal, and the outside receptors response.
But when we consider the electrical wiring of the brain, is it possible to use the heart as a simple model? You know, many ion channelists favor to choose the heart as the target organ for research, which contain simliarly functioning ion channels, but much easier for them to unravel the mechanisms underlying.
Or, given the gap junction thing, no way?
Well, I need to think more before I went up to ask him.
Janelia Farm
When I just came to CSHL, there're already several established scientists ruthlessly leaving the beautiful place. Where did they go? Janelia Farm. The post docs are so surprised when finding that I've no idea about this Holy Land of biomedical research. They can't help to describe how cool it is, which to me, was like a legend.
No wonder the most alluring thing is the full support from HHMI. You keep a small lab, you just need to stay in the lab, focusing on your experiments and search peer papers. No need to worry about funding, no need to worry about how can I my postdoc. You just need to do your science, pure science, completely science! And the transparent structure of the building construction, you see it? I love that! I have stayed in the department office for 1 year, you know how it's like, no windows, the only outlet is the door, small room, all walls surrouding, no difference in day and night, dead silence if you close the door and shut down all the machines, you know what's that like? A jail.
Oh, there's a 5 minutes documentation available.
Description about Jenelia Farm:
We aim to identify important biomedical problems for which future progress requires technological innovation and then we foster the establishment of integrated teams of biologists and tool builders who seek to break through existing barriers.
two synergistic areas that are particularly well matched to the Janelia Farm environment:
1. The identification of general principles that govern how information is processed by neuronal circuits
2. The development of imaging technologies and computational methods for image analysis
1. The identification of general principles that govern how information is processed by neuronal circuits
2. The development of imaging technologies and computational methods for image analysis
Graduate Program:
The Howard Hughes Medical Institute offers two programs leading to the Ph.D. based at its Janelia Farm Research Campus, in partnership with the universities of Cambridge and Chicago. These accelerated programs are designed for a small number of well-prepared, highly committed, and gifted students. We offer opportunities for interdisciplinary research in an intense environment.
People from CSHL:
Karel Svoboda
The marks building becomes so empty as he left. The good thing is the whole floor become Tony and Zach's. When I rotate there, I was always the only one on the first floor which originally was his lab. Heard of a lot about him, he is a very positive, active, and creative scientist, in everyone's eys.
Dmitri B. Chklovskii
I remember he do computational neuroscience.
Alla Y. Karpova
It surprised me so much when finding that she came here. Actually in marks building, her bench is just opposite to my bench. Funny enough that I met her first when I was doing experiments in one midnight, she gave me a shock when she appeared and went by quickly. Later I joined the farewell party for her.
Subscribe to:
Posts (Atom)
