Classifier domains of competence: The landscape contest is a research competition aimed at finding out the relation between data complexity and the performance of learners. Comparing your techniques to those of other participants on targeted-complexity problems may contribute to enrich our understanding of the behavior of machine learning techniques and open further research lines.

The contest will take place on August 22, during the 20th International Conference on Pattern Recognition (ICPR 2010) at Istanbul, Turkey.

We encourage everyone to participate and share with us your work! For further details about dates and submission, please see http://www.salle.url.edu/ICPR10Contest/.

SCOPE OF THE CONTEST

The landscape contest involves the running and evaluation of classifier systems over synthetic data sets. Over the last two decades, the pattern recognition and machine learning communities have developed many supervised learning techniques. Nevertheless, the competitiveness of such techniques has always been claimed over a small and repetitive set of problems. This contest provides a new and configurable testing framework, reliable enough to test the robustness of each technique and detect its limitations.

INSTRUCTION FOR PARTICIPANTS

Contest participants are allowed to use any type of technique. However, we highly encourage and appreciate the use of novel algorithms.

Participants are required to submit the results by email to the organizers.
Submission e-mail: nmacia@salle.url.edu
Meet the submission deadline: Wednesday May 26, 2010

The contest is divided into two phases: (1) offline test and (2) live test. For the offline test, participants should run their algorithms over two sets of problems, S1 and S2. However, the real competition, the live test, will take place during the conference. Two more collections of problems, S3 and S4, will be presented.

S1: Collection of data sets spread along the complexity space to train the learner. All the instances will be duly labeled.

S2: Collection of data sets spread along the complexity space with no class labeling to test the learner performance.

S3: Collection of data sets with no class labeling, like S2 to be run for a limited period of time.

S4: Collection of data sets with no class labeling covering specific regions of the complexity space to determine the neighborhood dominance.

For the offline test, the results report consists of:

1. Labeling the data sets of the collection S2.

The procedure is the following:

1. Train the learner using Dn-trn.arff in S1.
2. Provide the rate of the correctly classified instances over a 10-fold cross validation.
3. Label the corresponding data set Dn-tst.arff in S2.
4. Store the n models generated for each data set to perform the live contest on August 22. Be ready to load them on this day.

2. Describing the techniques used.

A brief summary (1~2 pages) of the machine learning technique/s used in the experiments must be submitted. We expect details such as the learning paradigm, configuration parameters, strength and limitations, and computational cost.

IMPORTANT DATES

* May 26, 2010: Deadline for submission of the results and technical report

* May 29, 2010: Notification of participation

* Aug 22, 2010: Release of S3 and S4

* Aug 22, 2010: ICPR 2010 – Interactive Session

CONTACT DETAILS

Dr. Tin Kam Ho – tkh at research.bell-labs.com
Núria Macià – nmacia at salle.url.edu
Prof. Albert Orriols Puig – aorriols at salle.url.edu
Prof. Ester Bernadó Mansilla – esterb at salle.url.edu

The contest will take place on August 22, during the 20th International Conference on Pattern Recognition (ICPR 2010) at Istanbul, Turkey.

We are planning to have a day workshop during the ICPR 2010, so that participants will be able to present and discuss their results.

We encourage everyone to participate and share with us your work! For further details about dates and submission, please visit The landscape contest webpage.

The advances in AI field, however, start to raise some serious concerns about robot autonomy and its social status as well as how to face this social disruption, and the three Laws elaborated by Asimov to protect humans from machines start to make sense for other than computer geeks. Scientifics are concerned about the “loss of human control of computer-based intelligences”, and the past February, the Association for the Advancement of Artificial Intelligence organized a conference in Asilomar (not a casual place) to discuss the limits of the research in this field. Development of machines that are close to kill autonomously are worth a discussion by those involved in the creation of the brain of such devices. The news of this event has leaked in the Markoff’s article in the New York Times.

On the other hand, who will be responsible for damages caused by these autonomous friends? Themselves or the corresponding designer? In this sense, philosophy should play a leading role in the design and integration of these “future citizens” since they should have a moral system allowing them to learn ethics from experience and people, and also find their place in our society. The latter implies to create a legal framework that defines machine’s civic rights and duties which is a proposal under study (see the news published by “El Periódico”, in Spanish language).

Finally, one may ask whether or not we are ready to live with human emulators. In my view, we are not. Although in the past years we have been skillful to adapt to new and challenging situations, and our experience with immigration integration and race conflicts should help us to welcome these new electronic neighbors, I tend to think that coexistence with robots will be one of the greatest challenges mankind has ever faced. Anyway, we will need to figure out the way to overcome it because the individualism and loneliness ruling our current society is leading us unrelentingly to a future with custom-made roommates.

For further information, please see the program.

This nice introduction helped me to follow the exciting talk Quantum computer compilers, performed by Prof. Al Aho, a computer science celebrity and one of the authors of the AWK programming language and of the so-called Dragon Book, Compilers: Principles, techniques, and tools.

The talk focused on the following six questions:

1. Why is there so much excitement about quantum computing?
2. How is a quantum computer different from a classical computer?
3. What is a good programming model for a quantum computer?
4. What would make a good quantum programming language?
5. What are the issues in making quantum computer compilers?
6. When are we likely to see scalable quantum computers?

Prof. Aho presented, with a clear explanation and a touch of humor, the fascinating field of the quantum computers by describing how “computation is just a particle dancing around others”, enumerating the four postulates of the quantum mechanics, mentioning that quantum teleportation is information transmission based on changes that take place instantly, envisaging programming without copy operation… However, despite the wonders of the quantum computers, it seems that we should wait a little bit more for being able to solve NP-hard problem.

Finally, take a glance at The Blog of Scott Aaronson. This is an unusual and interesting blog about this topic.

Its program, full of interesting talks that discussed the new trends for knowledge extraction processes by means of evolutionary algorithms, included two contributions from the GRSI entitled: Multiobjective evolutionary clustering approach to security vulnerability assessments and Beyond homemade artificial data sets, presented by Guiomar Corral and Albert Orriols respectively.

Guiomar Corral introduced an evolutionary multiobjective approach to cluster the devices of a network with similar vulnerabilities. This technique provides analysts with a map which is helpful to detect malicious attacks or unauthorized changes in the network.

Albert Orriols, in turn, addressed a hot topic in machine learning: the artificial data sets generation. He explained the importance to work under a controlled experimental framework and pointed some ideas to build it.

Salamanca will be for one more day a forum to exchange new ideas and present recent developments in the field of artificial intelligence.