New Sindarin Dictionary application

[Lúthien]

Suilad,

I have been working on a new version of Didier Willis' Dragon Flame / Hesperides dictionary application. I have created a page with some information about it on and also sent a message to Elfling and the German based Mellyn Lammath forum. It comes down to that I would love to have input from possible users: what they think this application would need in order to be useful.

If you have any questions I'm all ears to hear them!

===============================================================================================================

Suilad,

I have been working the past year on creating a new Sindarin dictionary application.
I've used Hesperides (the existing Sindarin dictionary application) a lot but it could use a brush-up. I approached the author - Didier Willis - and offered my help in creating some or other new version of Hesperides / Dragon Flame which he accepted.

The preparation took quite some time. This was mostly taken up by trying to get my head around the specific format in which the data (the word lists) were put; of thinking of a way to rework this to another format which I could work with and, lastly, then actually doing that conversion. If you're interested: the gritty details are below.

I'm now at the final stage: the point of creating the user interface: the way the program works, receives input and presents output.
It would be very helpful to have the ideas of potential users about this: what would you think of as an ideal dictionary application?

Even though its a pretty simple application, there are many choices to make that can make all the difference between really breathtakingly usable (like Skitch) or about as dumb as a sack full of frozen hammers and as usable as a virtual paper weight (I've encountered some truly great examples of that latter category in my job at a University Library).

It's about things like "what is a good way to search for a word?"
Maybe you have noticed that "auto-suggest" feature in Google .. maybe that would work well.
Or have a list with matches that's being populated as you type so that you don't have to click a "search" button (the current Hesperides can do that).
What search modes? Match the beginning of the word, *anywhere in* the word, or exact matches? Any other possibilities?
Would it be handy to be able to add your own constructed words? Would it be good to present those then with, say, another colour?
Or to be able to add your own private notes to dictionary entries?
Would it be good to have the Sindarin -> English (or French, or German) and the English -> Sindarin both alongside one another?
Maybe offer the possibility to choose large fonts?

Anyway, I'd be happy with all ideas and suggestions.

Also, if there's anyone willing to test the app, let me know. The program will run on any platform (mac, linux & windows - and some more even) and it's about 10 mb to download - that's including the database.


Thanks in advance
Garo hîdh,
-l

(technical stuff below here)

Didier told me that the word lists as he has them - both the ones on his Hisweloke website as the list that's in the existing Hesperides application - are in the "TEI" (Text Encoding Initiative, see ) format, which is an XML format designed to represent text; it is specifically geared towards use in linguistics, humanities and social sciences.
It is perfectly possible to use an XML based document as the source of an application - Hesperides works like that. But it has some drawbacks like a noticeable delay when the document loads - though this is typically one of those areas that can stir up arguments between pro's and contra's. For me it's been clear for years that my brains are somehow incompatible with XML, so if I were to do this, I had to rework the whole thing to a relational format.
It's been a fun challenge to first figure out what subset of TEI was actually used in the files that Didier had sent me and to remodel that into a database schema. To make a longer story short, Perl was a good friend there: and again in re-hashing the list to insert statements to populate the database with.
After that, I had a database consisting of some 12 tables that contained all the information from the TEI encoded word list.

This was actually the hardest part. Building an application that then accesses that database is quite straightforward. This week, I have built a little demo application that does nothing but give the whole list of words and some additional fields from the database .. translation, pronunciation, etcetera. It is built in Java, so it runs on any platform; and the data are stored in very lightweight embedded database (Apache Derby). The whole thing is about 10 megabytes now, database included.

It is also possible to create a web interface for the same database - something that you can put on a website so that users can access it online. An applet, or a php page, or Adobe Flex maybe if we want to make it really snazzy
It's not too hard to add additional languages to the available translations (english, french and german at this moment).
And should someone have a TEI encoded Quenya wordlist, there's no reason that that could not be integrated in the application as well - using some sort of switching between modes, maybe.

I've put some of the technical info like the DB schema etc. on - you can also download the app there (though it does not yet offer any functionality apart from scrolling through the word list and displaying some fields). It will only need a Java Runtime Environment, but practically all computers have that installed. If not, you can download it for free at

[Lúthien]

here's a second test version; this one starts out with the whole list, and then narrows it down as you type (the same behaviour as the old Hesperides app).

The match is now anywhere in the entry - so th matches _thamas_, _leithian_ and _pendrath_

[Lúthien]

update ...

I finally found time again to work on this. All my extra time until now has been put into a fundraising project my friend and I have been doing - but now that's behind us, I took up this project again. It really should be completed now, because it has been in limbo much too long.

I have done some more work on the database design and I think that I have it now firmly in my head how to finish this application.
It's now a matter of coding some more queries and figuring out how to display the results nicely with the proper fonts, like in Hesperides.
I also have to find out how to render the pronunciation column - it displays plain ASCII like "Ad"uj.Al" (Aduial). I suppose there is a special character set for that: I will have to search for how to do that properly.

I have now created a search screen which takes a wildcard search for a string and returns the Sindarin matches. When one entry is clicked, the rest of the information is fetched from the database and displayed.
The output is not yet formatted; it is displayed as it is fetched. I also have to add the "note" and "mentioned" parts plus the references, but now the rest is in place that should pose no problem anymore.

I am quite happy with the performance. The application starts in a few seconds, and record fetching is practically immediately.
Here's a screenshot. I have promised Didier (who helped me initially, and gave me the XML TEI files with the dictionary data) to finish it now as soon as possible.



PS - this example uses the dataset I created from the XML file that has the German and English translations only. For the final version, I will of course add the French translations as well. I took this smaller set in order to keep the (visual) complexity of the TEI schema as low as possible - and that already was complex enough

The idea is that you can select English, French or German. And it will, of course, also be able to search for the entries on the translations.

That should give a usable dictionary application, but I am planning to extend it with functionality such as:

- adding private notes to entries;
- adding entries manually;
- importing (updated) dictionary files;
- exporting to file
- printing out the whole dictionary or parts of it
- if I can find a TEI encoded Quenya word list - or indeed ANY structured format would do (Perl is patient ) I could also add Quenya to this application
- maybe it would be handy to include a stem search or list? This could be linked to entries that have stems mentioned

I only hope that I "keep going" now. Nothing stops me from completing it, except other interesting things. As usual

[Eryniel]

Great to hear this handy application is now back in development
As always, if you need someone testing, you have a guinee pig...

[Huanarmo]

Hannad, Lúthien.

That is phenomenal work. And, like Eryniel, I'm happy to be a 'test pig' too.

Garo 'lass

[Lúthien]

- merged the French data (translation, notes, etymology info) into the database
- building the GUI
- assembling the queries to retrieve the data to build the output with

Now all the data are consolidated into the database. It was a bit challenging to merge the Sindarin-English-French set into the existing Sindarin-English-German set because of some small, but significant changes - like some entries that only existed for French. Some French notes were so elaborate that I had to widen some text columns
But nothing really obstructing here. I figured out how to solve the differences and it seems to be consistent. Doubtless we'll find *some* errors, but they can always be fixed later on.

I hit upon something I did not understand, though. Didier Willis had encoded the pronunciation information into a format that did not make sense to me:

Here's an example:

<entry id="niphredil">
<form type="regular">
<form>
<orth>niphredil</orth><usg type="lang" norm="S."/>
<pron></pron>

and this is what this entry looked like, rendered in Hesperides, the old application:

niphredil* [] (nifredil) n. Bot. a pale winter flower, snowdrop ◇Ety/376, Ety/378, LotR/II:VI, Letters/402, X/PH ◇ niphred+-il "little pallor"

I am now wondering how to map this <pron>n"ifrEdil,</pron> to this [nˈifrɛdil̡] ...
I think that I do understand why Didier did not just put the IPA characters in the TEI schema: I've experienced it often enough that XML is very picky where it comes to what it considers valid characters, so maybe it does not accept some of the IPA characters.

So far, it seems that it is a 1 to 1 mapping: e = 3; θ = T; ð = D ... but I have no idea whether he created this himself or if it is a standard encoding. There also seems to be a link between capital letters and stress, but I am not certain.
So I asked him, and also someone from the University of Amsterdam, of whom I found an interesting page about IPA characters in Java - just what I needed.

I have had answers from both, and I still need to research that. It comes down to that what I thought is indeed the case: it is an ASCII mapping of the IPA set, or rather, a small subset of it. But large enough for all the Sindarin sounds, apparently.

This is how the application looks now:






as you see, you can choose one or more "output" languages. Eryniel said she'd love this multi-language output possibility, because of the fact that translations are often not really one-to-one. Words do not always cover the same notion, and having additional information in the form of more than one translation might help determining the right choice.
But there is no need to use it, of course.

The input language - on what "index" you search if you search in a modern language - is set in that dropdown-list (combo-box) top left, after "search".
Changing this changes the button next to Sindarin, and also all captions that are now in English:



If I ever get around to adding Quenya to this application, I can add a similar switch on the elvish side of things.
Note that the output is still completely unformatted. After that's done it will look like hesperides' output:



Also, the list you see at the left (the search list) is also still "dirty" - there are entries in there that shouldn't be there, because they are reference targets. Anyhow, 't will be fixed

[Lúthien]

I just had to add this .. I looked up one of those "long" French notes - it turns out, that the German is equally lengthy!

[Lúthien]

Hey, I can embed a Jing screencast .. this opens up possibilities.
Anyhow, today's state of the Sindarin dictionary app.
Getting there!

[órerámar]

Lúthien,

I am in awe, I have no words. Maybe "maira" is appropriate! (maira adj. “admirable, excellent, precious”; “splendid, sublime” (“only of great, august or splendid things”). If this is not splendid, what else is?

You are a "*nonwamaitar"

[Lúthien]