Proposed mentor: Xin Li <delphij@FreeBSD.org>

The libiconv library is an important piece of I18N software. It implements library routines to convert files from one encoding to another. There is also a command line interface for this library, which provides a convenient way of encoding conversion. Currently, FreeBSD lacks of these pieces of software in the base system. In the Ports Collection, we use GNU libiconv but this is such an underlying tool that we should have an own, customized version that is:

Let's summarize the benefits once more:

There have already been two efforts to implement a BSDL libiconv. One of those has been developed by Konstantin Chuguev and Alexander Nedotsukov. It can be found in the Ports Collection as converters/iconv. This will simply be referred as "BSDL iconv". The other one is integrated into NetBSD and it comes from the Citrus project.

As for the first mentioned implementation, the missing features or known bugs are:

I have to highlight that despite its deficiencies, this implementation is very well-documented and the code is easy to read. It serves as a good reference of conversions.

The second existing implementation is part of a wider project, called Citrus (Comprehensive I18N Towards Respectable UNIX Systems). It is quite mature and more complete than the aforementioned one. This one should be chosen as a starting point. The objective is to extend this implementation to support all the character encodings listed in the section called “Project Description” and to solve possible issues found during the work.

Here is a table that summarizes the encoding support of the two implementations:

I plan to work around 4 hours a day on the project. In July, I will have some vacation to Spain but during this period I also intend to dedicate some time to the project. If I have time before, I want to start the work earlier, in the community bounding period because I am already familiar with the community and the infrastructure.

Here are the concrete planned TODO items and their schedule:

During the development, GNU libiconv's regression test suite can help to track correctness of various encodings (this test suite was used to obtain the results listed in the table above). An own test suite would be good but I prefer to concentrate on the real work instead of writing test suites and the GNU test suite seems good and thorough enough for testing.

As for the performance, it is not the most important factor for libiconv because the conversion of encodings is usually not a time critical task but of course, an acceptable performance must be provided. This can be tested by comparing the conversion time of Citrus and GNU on different type of conversions with different file sizes.

At the end of the program, I would like to test the created library with a complete portbuild run. This project will not provide a completely GNU-compatible implementation, because the implementation os such great many of encodings cannot fit into the timeline of Summer of Code, so some ports may break with it but checking the build logs can help identifying the possible bugs.

Apart from this test plan, the library will be accessible through Ports Collection once the basic functions work so that interested parties can try it out and provide some feedback.

During Google SoC 2008 I worked on a BSDL implementation of grep and sort. The former is ready and waiting for a portbuild test. I think chances are good to import this before FreeBSD 8.0. The latter is also nearing completion. Both utilities are widechar-clean and offer a good performance.

As for the I18N area, I learned some basic things during Google SoC 2008 because widechar-support was a crucial requirement for grep and sort, so I read various documents about encodings and I got interested in the topic. In this year I would like to obtain more knowledge on encodings.

During Google SoC 2007 and 2006 I worked on various Ports Collection infrastructure-level enhancements. Besides, I work as a doc/ports committer and I was the first Hungarian translator, who launched the www/hu and doc/hu_HU.ISO8859-2 trees. I also mentored some committers in the ports/doc area.

My CV is available online in the following formats: HTML, PDF, RTF, PS, plain text.