Incremental Sigmoid Bayesian Networks

[python obj model  time   01:09:10]

Incremental Bayesian Networks for Natural Language Parsing

talk at Google Tech Talks

James Henderson, U Geneva

ISBN's provide a powerful method of feature induction

This talk reminds me that I have a lot to learn about statistical processing and machine learning.

new terms: 

marginalize 

... related to to summing over all data, which is avoided

fully factorized

... without any links

beam search

... look at 100 best options, with each new word

branching factor

... limits blow up

Has ability to pass features.

Simple Synchrony Networks (Henderson 2003) are claimed to be ä strictly feed-forward approximation equivalent to neural networks (presumably back prop).

The means in mean field approximation turn out to be equivalent to the activation value of an edge in the neural network. Using discrete (0-1) random variables.

Perhaps hidden variables and visible variables captures the intuition about structural and substructural (substratal? features implicit from lexicon) analysis.

Using models with vectors of typed features, rather than trying to induce a grammar on atomic symbols.

Software

SSN Statistical Parser: A broad coverage natural language syntactic parser. 
ISBN Dependency Parser: The statistical dependency parser described in [Titov and Henderson, IWPT 2007] and evaluated in [Titov and Henderson, EMNLP-CoNLL 2007]. 

I.Titov and J.Henderson. A Latent Variable Model for Generative Dependency Parsing. In Proc. International Conference on Parsing Technologies(IWPT 2007), Prague, Czech Republic, 2007.

I.Titov and J.Henderson. Fast and Robust Multilingual Dependency Parsing with a Generative Latent Variable Model. In Proc. Joint Conf. on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL 2007), Prague, Czech Republic, 2007. (CoNLL Shared Task, 3rd result out of 23)

James Henderson, Peter Lane

A Connectionist Architecture for Learning to Parse (1998)  (8 citations)

Dept of Computer Science, Univ of Exeter  PDF