1. Conx Backprop Python Package

NEW VERSION! 6.0 Updated Mar 10, 2003

1.1. Overview

This site describes Con-x, the Connectionist Backprop Language and Simulator. The idea behind this system is to allow experimenters to quickly and easily create, train, and test feedforward, backprop neural networks. Basically, Con-x (pronounced "kun ex") is a neural network scripting language and environment.

Who's it for? It was designed to be used by serious backprop researchers, as well as a teaching tool for use in introductory AI courses. There are many simulators out there, and many concentrate on the novice student. Unfortunately, these systems usually sacrifice advanced functions (such as flexibility and ability to process large datasets) for their ease of use and nice graphical interface. Con-x attempts to create a tool which is easy to use, but one that the user will not outgrow easily.

Con-x works quite well with very large data sets, but has a simple scripting interface. We have used an earlier version inside a fuzzy logic/ artificial neural network robot controller (See papers  here and also  XRCL). All sources and related files are distributed under an free, open-source license (GNU GPL).

1.2. What's new

Everything! This is a complete rewrite of Con-x in Python. The old version, written in C++, will no longer be maintained. You can find it  here.

Python is a scripting language that runs on most platforms, including Linux, Unix, and Windows. I have ported the old Con-x ideas to Python. Now, I'll let Python handle the language issues, and concentrate Con-x on the Neural Network issues.

Con-x is now part of a larger artificial intelligence and robotics project called Pyro.

What might this be worth learning Python? Well, you can now have multiple networks, each in their own thread (if you wished), graphics, and a host of other before unimaginable tricks. And it is very easy. For example, I took this version of Con-x, and created a way of evolving networks in an evening. See the 'ga' class of Pyro  here.

At what cost? Yes, this version is a bit slower than the old C++ version. But it would not surprise me to find a Python->C compiler someday. And the advantages of the new version far outweight any speed decreases, I think.

You don't need to learn much of Python to get going with Con-x. Here are some samples. Pay attention to indenting. You can load these from a file, or enter them interactively and then save the resulting network to a file for later running.

1.3. Samples

The first sample is the standard XOR problem. This code creates a 2-2-1 three layer, feed-forward backprop network.

    # Con-x: Sample XOR Network
    # (c) 2001, D.S. Blank
    # Bryn Mawr College
    # http://dangermouse.brynmawr.edu/
    
    from pyrobot.brain.conx import *

    n = Network()
    n.add( Layer('input', 2) )
    n.add( Layer('hidden', 2) )
    n.add( Layer('output', 1) )

    n.connect('input', 'hidden')
    n.connect('hidden', 'output')

    n.setInputs([[0.0, 0.0],
                 [0.0, 1.0],
                 [1.0, 0.0],
                 [1.0, 1.0]])

    n.setOutputs([[0.0],
                  [1.0],
                  [1.0],
                  [0.0]])

    n.setReportRate(100)

    n.reset()
    n.setEpsilon(0.5)    
    n.setMomentum(.975)
    n.train()

The Layer() class constructor takes a name, and size. The network connect() method takes layer objects.

Here is a Simple Recurrent Network:

    n = SRN()
    # Sequence is automatically used when the input pattern is
    # larger than the input layer: 1 input, but input pattern has
    # more elements
    n.add( Layer('input', 1) )
    n.add( Layer('context', 5) )
    n.add( Layer('hidden', 5) )
    n.add( Layer('output', 1) )
        
    n.connect('input', 'hidden')
    n.connect('context', 'hidden')
    n.connect('hidden', 'output')
        
    n.setInputs([[0.0, 0.0],
                 [0.0, 1.0],
                 [1.0, 0.0],
                 [1.0, 1.0]])
        
    n.setOutputs([[0.0],
                  [1.0],
                  [1.0],
                  [0.0]])

    n.setSequenceType("random-segmented")
    n.setReportRate(100)
    n.setResetEpoch(10000)
    n.setTolerance(.4)
    n.reset()
    n.setEpsilon(1.5)    
    n.setMomentum(.9)
    n.train()

To save your network, try this:

    n.saveNetworkToFile("myfile")

Your network will be saved as a Python file in "myfile.py".

To test a trained network:

    n.setLearning(0)                   # turns learning off
    n.setInteractive(1)		       # turns interactive on
    n.sweep()                          # run through the data

Here is an example running:

$ python2 pyrobot/brain/conx.py 
Do you want to run a BACKPROP network? y
 Backprop: .............................................
Epoch #   100 | TSS Error: 0.98 | Correct = 0.0
----------------------------------------------------
Final #   126 | TSS Error: 0.45 | Correct = 1.0
----------------------------------------------------
Do you want to run a QUICKPROP network? y
 Quickprop: ............................................
----------------------------------------------------
Final #    46 | TSS Error: 0.33 | Correct = 1.0
----------------------------------------------------
Do you want to train a sequential XOR SRN? n
 Do you want to see (and save) the final network? y
 Filename to save network (.py): test
 Step # 1
Display network 'Backprop Network':
 Display Layer 'output' (type Output):
  Target       :  -0.5 
  Activation   :  -0.365042686462 
  Error        :  -0.139404565096 
 Display Layer 'hidden' (type Hidden):
  Activation   :  -0.203501075506 -0.253453046083 
  Error        :  -0.168179109693 -0.16878734529 
 Display Layer 'input' (type Input):
  Activation   :  -0.5 -0.5 
--More-- [ENTER]

1.4. Download

You will need:

1. Python 2.0 or 2.1
2. Numeric Python module (numpy)
3.  Con-x

Once you have Python and the Numeric Python module installed, create a directory named 'pyro', and another 'pyrobot/brain'. Put the Con-x source file (conx.py) into 'pyrobot/brain/'.

You now only need add the directory that pyro is in to the Python search path. Under Linux, that might look like:

export PYTHONPATH=$PYTHONPATH:/home/userid

You should now be able to run Con-x with:

$ python2
Python 2.1.1 (#1, Aug 28 2001, 19:51:39) 
[GCC 2.96 20000731 (Red Hat Linux 7.1 2.96-85)] on linux2
Type "copyright", "credits" or "license" for more information.
>>> import pyro.brain.conx
>>>

Now, you can interactively enter commands, as above.

1.5. What's New

1.5.1. Version 6.0

1. Removed Quickprop extensions (got too complicated to maintain)
2. Lisa Meeden and Doug Blank rewrote main Backprop stuff
3. It appears to work correctly

1.5.2. Version 5.9

• support for freezing weights
• support for setting layers active/inactive

1.5.3. Version 5.8

• bug fix: when learning was off, wed and bed were still being updated. This caused problems when learning was turned back on.
• added default values for copyActivations() and copyTarget() if the incoming array is too small.

1.5.4. Version 5.7

• loadWeightsFromFile(), saveWeightsFromFile()

Related topics:

Go here for full search functions