MIST Machine-Translation Evaluation 2006
November 17, 2006, 2:06 pm
Filed under: Information Tech, Metaverse, Surveillance

The (American) National Institute of Standards and Technology (NIST) makes yearly evaluations of machine translation technologies. The evaluations are based on Arabic – English and Mandarin-English translations. This year’s results are commented upon by MIT’s Technology Review. The results aren’t spectacular but there is concrete progress with Google and IBM jostling for top spot.

It would be great to see the real time translators used in Second Life where Yossarian Seattle has already provided the tools. There’s a good article on Yossarian’s translator at New World Notes and a video of it in action on YouTube.


2 Comments so far
Leave a comment

It would be very interesting to see the results of evaluation for other languages, especially for European (French, German, Spanish, Russian). As for the methods and the ratings (the best results does not exceed 0,5 – it means that less than a half of the text was translated correctly? In fact, the leading MT systems, like Systran, PROMT, T1 provides up to 80% of correct translations), I guess, it would be more efficious to make evaluation by human translators who know the both languages, and to count the number of mistakes in such translation (perhaps, assigning different weight to each mistake). Surely, this way is more long and expensive, but it would give a better conception of the translation quality.

Comment by Elena Temnova

I agree entirely. The NIST sponsored studies have a clear strategic significance – their measurements look like they are designed to satisfy the requirements of administrative decision procedures rather than specifically academic concerns. Your method is a far safer way to assess translation than by mechanistic means.

I must admit – I am as interested to see the systems you mention rolled out in real social contexts as I am to see them make the incremental steps towards maximum performance. I mentioned the SecondLife context because it provides the opportunity for users to adapt to the limitations of the translation methods.

Comment by autoassemble

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: