How is WebBeagle for a name ?

(Or FireBeagle ?)

More...(drag mouse)

Close Tab

Forward

Next Tab

Reload-Stop

Close Tab

Back

Home Page

I always wanted to search beagle using a web-browser. My desktop searches are mostly infrequent but complicated. So if I am unable to find something I definitely need an index-based search tool (e.g. beagle) but on the other hand I don't want to keep an application open on my desktop for a long time (beagle does not consume significant memory for me, so it can keep running in the background). That was one reason why I wrote the kio slave for beagle (I have stopped liking it for a long time) and followed it with a kde deskbar like applet (which is nice but with limited functionality). Now that I am done with shameless advertisements, let me share with you yet another way to query beagle.

Today I wrote some ajaxy,xslt-ed webpages to allow users (read: me) to query beagle using a browser (err... firefox, konqueror-3.5.5 does not have XSLT processor). It uses the networkservice backend that can be used to query beagle over network (based on last year's Google Summer of Code projects).It uses internal knowledge of how queries are serialized, lists all the information in a boring way and does not show snippets (yet). But it works and was reasonably fast in displaying 42 results. Besides the boring UI, allowing browsers to access services always opens up some security hole, so it is disabled by default. If you so desire, use it but at your own risk (check the commit log for how to turn this feature on).

I do hope to get this feature properly implemented. The things need to be done are:

1. Fix the network-backend (it is sufferring from some crashes).
2. Probably related to the network backend as well, some kind of search authorization is needed.
   
3. Use CSS + javascript for the results page to group/sort the results, and make them look decent.
4. Get snippets and display them. I am thinking of retrieving snippets only on demand. Somehow the name and the location of files or the sender and the subject of emails help me more in filtering out search results than snippets. Which is quite unlike how I use web search engines.
5. Figure out a way to use the C# or libbeagle API to create the xml request messages. Currently they are hardwired. If the solution turns out to be too complicated, it might not be a bad thing to leave the format hardcoded as it is now.
6. More cosmetic, separate command line and configuration options for this feature.
7. This is more related to the network query implementation; figure out how to use the QueryDomain thing meaningfully. The results from some backends only make sense on the same machine and even for some of them, it is tricky to open the applications just from the URI itself. So does it make sense to show e.g. evolution mail hits in the browser ? Does it make sense to return gaim (pidgin) hits when queried over network ?
8. How does the browser behave (read: choke) when it receives 1000 results (100 results each from 10 backends)? (i.e. the DOMParser has to parse a huge string and form a huge DOM of 1000 Hit nodes. You see why I dont like to get all the snippets beforehand :)?
   

Hey Firefox, Beagle This ... NOW!

Consider this scenario: B was writing an article when he remembered that he read a relevant blog some weeks ago but did not bookmark it due to his lack of foresight (stupid B). What to do ? He could possibly run a grep in his Firefox cache folder hoping that the cache would still be there or scan his FF history for a familiar looking URL. Anything better ?
If B was using the FF beagle extension, he could find the blog instantly

Really! Then I will pose this requirement. B has been using the new FF beagle extension for some time now. But he does not want to index all his browsing history. Sometimes he just wants to browse off the record. Maybe after a while he wants to index the pages he visits. Pretty random ... but hey, that's B's nature.
No problem. The new extension puts a small status icon, which can be  used to disable/enable the automatic index; really, at B's own sweet will.

That was easy. How about this one, a bit harder. Someone told B about FF beagle extension, and B started using it. But this time he needs a PDF paper he found online. He does remember that he read the paper and he must have read it in the browser itself. Now what ?
If B was using the new FF beagle extension, he could even search for embedded documents. Embedded content is not indexed automatically (as of now) but can be indexed as required using the status icon menu.

Hmm... But what if B visits a page with auto-indexing disabled and then decides that he better index the current page. Now what ? Does he have to enable the auto-indexing and then reload the page. Thats stupid!
Heh. You gotta be kidding. Of course he can use the status icon menu or the page context menu to index the current page.

If B wants to index an image, or take this one, sees a link to something (PDF, image, java source file, mp3 ...) and wants to index it without loading it explicitly ?
The context menu provides options to index an image and index a link; in both the cases the file is retrieved in the background and indexed. All filetypes that beagle can filter, can be indexed using the index this link feature.

This new extension seems to uber-useful. I guess I err... B should really start using it. Can B save notes while indexing on demand i.e. if he is indexing a link he might want to index a few extra keywords (for querying or for reference) ?
Yeah! B gets that too. If B checks the corresponding preference entry, the extension will prompt for extra keywords to save and index when B explicitly asks it to index something.

I will be good and share this new beagle extension with everybody, written by Tao as part of Google Summer of Code 2007. Soon to be found in beagle trunk.

Mr Beagle, you are on the air

Back in India, I did one good thing. Finished adding Alexis Christoforides' project of Google Summer of Code 2006 Beagle over the network. Its a bit untested with the recent versions of beagle and I didn't read the code completely, so I left it in a separate branch for the time being.

What can you do with this wonderful project ?
- Start beagled on one machine
- Add that machine (with port 4000) to the list of neighbourhood nodes via beagle-config on a second machine in the network
- Remotely query the beagled on the first machine from the second machine

The SoC project was a complete one, with GUI support. However, I only added the basic infrastructure and enabled the cli tool beagle-query to query remotely. I am not sure if querying for emails remotely makes sense, how will the results be opened ? As of now, only the filesystem backend is enabled to send query over the network.

Ideally, beagle should automatically find out other networked beagled in the network. Don't despair, there was another SoC project in 2006, Avahi Integration by Kyle Ambroff doing exactly that. Sadly I am too preoccupied to work on integrating the Avahi work. But with that and the GUI parts of Network Beagle, beagle would be useful in more ways than ever.

Silence of the Dog

Lately beagle releases have slowed down quite a bit; there were a few bug-fix 0.2.16.x release and another 0.2.17 bug-fix release (it was supposed to 0.2.16.4, but the changelog was too large for a point release). The underlying goal is to get ready for 0.3.0; svn trunk is changing so rapidly these days that it is difficult to isolate the simple ones and make them into a 0.2.x release. On the other hand, the changes are too major to be put into a 0.2.x release (they would also need extensive real life testing).

Recently I moved beagle to use taglib-sharp for filtering music files from entagged-sharp. I was told entagged is no more actively maintained and taglib is definitely seeing a lot of rapid development. My timing was not quite right, the 4th March news "Entagged is unmaintained" is followed by the 28th March news "Entagged is maintained". I came to know about it only after I made the transition. Too late! On the plus side, taglib# has support for larger number of formats and is being used by Muine and Banshee, so expect sharing of taglib-sharp libraries. Unfortunately, there are no taglib-sharp packages out there yet (there is a proposal for a debian package), so all the mono apps are currently including taglib-sharp by its source. We too initially source included it, then removed it and instead linked against the package. But if there are no packages for the major distributions, it might make sense to source include it; compiling Beagle is pretty demanding anyway.

In other news, I used the extremely handy heap-shot to identify that instances of IndexReader were not being GC-ed even long after the corresponding method ended. Explicitly setting them to null immediately freed them. I suspect some thread local storage magic happening behind my back. Note to self, set IndexReaders to null immediately after they are closed. Did I say heap-shot is amazing ?!

There are several more improvements to the speed and memory performance of IndexHelper and BuildIndex. One notable feature I added was to reduce re-indexing of files which could not be filtered before. Due to the inherent distributed nature of beagle indexing, the crawler is always separated from the indexer. So if the crawler finds some file which was not filtered before, it has to re-submit it to the indexer. Who knows! There might be a suitable filter now. The downside was that a lot of files were being repeatedly re-tried by the indexer, slowing down the whole process. I decided to store the files containing the filters and their last modified times in a filterver.dat (akin to mozilla pluginreg.dat) and if the filters were not changed since last run, assume that there is no newer filter. Fair guess I would say.

Beagle knew how to index email attachments for quite some time; some months ago it also got the ability to index archives. However all along this was done by extracting the included files to a temporary file and then indexing it. This was done primarily because of the way included content (aka child indexables) were handled and also due to the fact that some of the filters only worked on physical files and not streams. This whole temporary file business never pleased me, there were race conditions which could leave undeleted temporary files in the system, even small included files had to be written to disk and further, extracting the contents of an archive to index it defeated the whole purpose of archiving it. Last week, I added the infrastructure to allow indexing of archives and email attachments without extracting them, if the filter permits of course. The infrastrusture is there, the archive and email filters should be modified to take advantage of this.

Finally, one feature I personally would like to see in 0.3 is support for XMP sidecars. XMP sidecars allow users to add a separate file.ext.xmp file containing arbitrary metadata (but in the XMP format) about file.ext. Really extensible solution for metadata. The main part of the code is in svn trunk; it still does not support renaming or deleting xmp files. Hopefully it will be finished in time.

This will probably be my last post before my annual break to the land of mangoes (" fruit of the gods"). Sadly, I have (knowingly) only tasted about a dozen varieties of mangoes, out of over 300. I will definitely try to increment the number this time. Next post, July.

Upgrade to Spring

Yesterday in a fit of mind I decided to upgrade to Mandriva latest release.
The steps included
- backing up .kde, .kderc, .qt, .gtk* and .local
- logging out of kde
- setting up a mirror as a distribution source for urpmi (mandriva is offering
non-free e.g. sun-java in its free source these days, but I still need the
PLF sources for codecs, BCI enabled freetype and fontconfig and a few other
things)
- # urpmi urpmi
- # urpmi <bunch of> kernels
- # urpmi --auto-select
and then selecting the ones I would like to upgrade from them

Soon I was running Mandriva 2007.1 Spring (Free). Yay! Its beautiful. The
Ia_ora theme and other Mandriva artwork is gorgeous. I legally own a XP cd,
from which I extensively use Verdana (for text) and Tahoma (for widgets).
They look wonderful as always with plf freetype (w/ hinting). I like to use
large fonts, enough to be readable 4 ft afar but somehow the deja or
bitstream fonts have a weird fuzziness in the curves of 's' and 'o's. I
cleared the settings of a test user account and a default new account looks
quite good (apart from the kbfx-styled mandriva menu).

KDE was upgraded to 3.5.6; I was actually using a few kde-3.5.6 packages from
cooker so there was no huge surprise. I was worried that mandriva would mess
up some of my settings when I log in for the first time as my normal user but
thankfully that did not happen. The system feels faster, konsole definitely
starts faster than before. Overall, I am extremely pleased with 2007.1; I
wish them all the best.
http://wiki.mandriva.com/en/Releases/Mandriva/2007.1/Tour

Some minor annoyances:
- mandriva kernel (based on 2.6.17) still has the weird cpufreq bug where
scaling_max_freq is same as scaling_min_freq (thus rendering all the
governors useless). It is probably the same problem described in

http://www.mail-archive.com/linux-acpi@vger.kernel.org/msg04484.html
- Suspend to RAM is broken in mm kernel (Mandriva has moved to pm-utils and it
works like a charm w/ the default kernel)
- tmb kernel has some problems of high CPU usage when copying files and broken
resume from s2ram. I still need to test the other tmb versions. I really like
the tmb kernel improvements.

Theory Guy...

This is not really techie but has a somewhat tech-vs-theory war flavor. The following list came out of today's departmental Friday lunch. Top 10 "ways to tell a theory person from a systems person":

10. One gets job and another does not.

09. One likes 10000n^2 better than n^3.

08. Space-time is important outside Star-trek.

07. In therory they are same, but in practice they are not.

06. P is not equal to NP divided by N.

05. n^10 is efficient.

04. SAT (Famous NP complete problem also known as Satisfiability) is most of the times easy to compute.

03. (Specific to our department) One can be only be found across the street.

02. Its pronounced "Lee-nux" (not "Lai-nux")

01. Computers do more than just email ?!

TinyBeagle or a Lucene Example

Recently I read this interesting comment in an OSNews article. It tried to briefly summarize what beagle is. I take users' comments very seriously and this person seem to know some internal of beagle, so I thought maybe he is true (modulo some factual errors). Maybe the only new thing in beagle is the crawler, the GUI and the scheduler; its mostly little C# glue code tying up a few third party apps.

So, I wrote down a small Lucene.Net based file indexer and query program. You index by
mono LuceneLocate.exe /path/to/index/dir index /directory/to/index
and query by
mono LuceneLocate.exe /path/to/index/dir query query_term
Pretty simple program, 85 lines of actual code. Incredibly fast performance. Using external program ('cat') to index files in a directory (recursively), it indexes 180 files in 0.06 seconds. Query returning 44 results took 0.0015 seconds . Takes 24 MB virtual, 5.3 MB RSS-Shared. No GUI yet. I could have added a scheduler to pause for 10 seconds after every 10 files (5 more lines). This Lucene.Net based crawler and indexer beats beagle in performance but nowhere close to beagle.

Maybe beagle is not a lucene-powered locate. After all, to err is human.

And PLTC means ...

I just now saw the programmer's personality test webpage in Digg and decided to give it a curious shot. And ... my programmer personality type is PLTC.

(Update: The link to my personality doesn't work. I thought they have one static page for each possible combination. I am probably wrong, let me add the full text here)

You're a Planner.
You may be slow, but you'll usually find the best solution. If something's worth doing, it's worth doing right.

You like coding at a Low level.
You're from the old school of programming and believe that you should have an intimate relationship with the computer. You don't mind juggling registers around and spending hours getting a 5% performance increase in an algorithm.

You work best in a Team.
A good group is better than the sum of it's parts. The only thing better than a genius programmer is a cohesive group of genius programmers.

This one I like the best.
You are a Conservative programmer.
The less code you write, the less chance there is of it containing a bug. You write short and to the point code that gets the job done efficiently.

OMG. That's totally like me. Or whatever!

kBeagleBar is alive!

I did not guess that people still use kBeagleBar, there are packages built for it and there was even an article about it and other KDE beagle search tools. Nice!

Faceless Bugs and Advanced Users

These are really two very different topics but they came to my mind while reading about Linus' Gnome patches and bugs.

The first is about creating a new account when I need to report a bug or submit a patch for some software. Most of the projects prefer attaching to bugzilla or they send it to their member only mailing list. I am extremely reluctant to create new accounts, so I have created bugzilla and mailing list accounts for KDE and Gnome. That covers a lot of ground. But still now and then I face a need to send something to somewhere else and bam! Sign up for an account sir! There is definitely merit in this approach, since otherwise bugzilla and mailing lists would be flooded with spam. But it definitely keeps me from submitting patches or commenting on something due my lack of interest in new accounts. Last week, a friend of mine (the inventor of Sperner's Game) was trying to install Kubuntu in his brand new Lenovo T60 when he spotted some typos in the installation windows. He was ready and willing to file a bug in Kubuntu and was told to create a new account for kubuntu bugzilla. As always, he was supposed to get a confirmation email.

The email came 12 hours later and I do not know if the bug was ever filed! Even if the email was prompt, the desire to report a bug has to be high enough to cross these technical potential barriers. *sigh*

This week I made extensive addition to beagle query syntax. There is an open bug in bugzilla asking for a visual way to add these advanced query expressions in beagle-search. I was thinking how best to achieve that; it is not easy to capture the power of beagle query expressions in a gui. I found the answer while reading some posts in desktop-architect mailing list about Linus' patch. There is nothing like an expert user or a novice user. Users always try to act as if they are smart and take the path of the expert user. Presenting different set of options for these different class of users does not work in practice.

In a similar style, there is no need for a GUI for advanced query expressions. Novice users i.e. users who will simply enter search terms will never know what a full boolean query expression does (with those OR and excluded expressions). On the other hand, expert users who know how to deal with the boolean expressions, the different keywords to do property search and other advanced syntax can anyway write it by hand. In fact, it is much easier for them to write it by hand than to do it visually. In this matter, I like the approach taken by
Google. I think I will push towards a simpler advanced search UI for beagle-search and Kerry, some simple choices like choosing type of file, extension, date range etc. Write the query by hand if you need that extra ounce.

beagle memory usage

Setup: Fresh run of beagled with only the kmail backend. IndexInfo report about 13700 items i.e. mails and indexed attachments. Beagle version is post 0.2.16, so that includes the individual items in the archive attachments as well. I started beagled as exercise_the_dog, indexing finished within an hour and this is the state after indexing is over.

VIRT      RES   SHR     COMMAND
--------+------+------+--------------------
167m    55m  11m     mozilla-firefox
248m    29m  2856    X
137m    20m  15m     amarokapp
72812   19m  6884    beagled-helper
89560   18m  13m     kmail
35816   15m  11m     konsole
42780   15m  14m     konqueror
49088   12m  5860    beagled
40320   11m  9524    kdesktop
42004   11m  9.9m    basket
32620   10m  9588    kmix
43896   9252 6220    kicker
37904   5884 3908    kded
34560   5600 2680    net_applet

Remember the rule: an approximate idea of the memory usage is given by RES-SHR.

beagle:Eat less, talk less be smart

Yesterday, beagle 0.2.16 was released. A couple of weeks back, we released 0.2.15 but I did not write about it. 0.2.15 came with a lot of performance and memory improvements, new backends, new features, lots of important changes . In the process, it also broke a few things. Those were fixed and 0.2.16 is a purely bugfix release for 0.2.15. I am considering 0.2.16 the best ever beagle release. Incidentally, 0.2.13+ releases somehow or the other had some nasty problems.

Combining 0.2.15 and 0.2.16, these are the major improvements:

* Very important, the looping bug is fixed. I would even like to claim, fixed forever. I happened to find some important clue while scanning the logs and other information provided by some of our very friendly and helpful users. Eventually our 3 year old database schema was found to be incorrect. Joe finally cleared the mess. Thanks Brian and Rick! This also means an end to the "log file filling hard disk" or "beagle indexing even after a week" type problems.

* Beagle uses some external tools to filter files e.g. pdfinfo, pdftotext, mplayer yada yada. These programs are well written and almost always work. Except some very malformed or wrongly detected mimetype file is sent to them and they go berseck taking up insane amout of memory or CPU time. Since the early release, we used to maintain that there is no way we can control the external processes. After all, we just use 'em. Joe finally put an end to that excuse by using some smart rlimit tricks to limit the resources used by these external processes. We still cannot control how mplayer might behave if given an word doc file, but if it behaves badly it will be killed before too long.

* Indexing data is a strenuous job. Think about all those heavy applications which process or generate these files. But people want indexing to be as silent as possible. There are frequently recommendations that beagle should use high nice, low system priority. low IO priority etc means to be as unobtrusive as possible. The fact is, beagle already does that. However, now we even go one more step by using SCHED_BATCH scheduler policy.

There are other side improvements too, RTF filter is new. The current one is based on the legendary RTF parser by Paul Dubois. Image filters are almost new; we now have Konversation (KDE IRC client) and KOrganizer (KDE tasks and eve nts scheduler) backends. By the way, soon after 0.2.16 was released, Opera webhistory backend was added to trunk. You can just drop the binary from here into your 0.2.16 /usr/lib/beagle/Backends folder and start using it, err... trying it. I do not know how complete it is.

I would like to end by thanking the excellent user base that beagle has developed. Without them, it would not be possible to fix a whole lot of these problems. Beagle would not be what it is today without them.

Seekable LineReader

Recently I need a way in StreamReader to get the position of a line which can be stored and later skip to that line directly. C# StreamReader API does not have any way of doing this, except calling Read() repeatedly and then doing the processing yourself. Which is clumsy. Note that, StreamReader.BaseStream.Position might be wrong due to underlying bufferring. I thought this should be a fairly common requirement and indeed, many people have same question on google groups or other forums. No good answer though. One reason is, such a thing does not really make sense for arbitrary Streams, since it might not be possible to seek in them.

I needed such a thing desparately, so I created an interface:

namespace System.IO {

 // A linereader interface
 public interface LineReader {

   // Returns a position marker, which can be used to navigate the lines.
   // Some implementations may only allow moving in the forward direction.
   // Might be different from line number or file offset.
   // Should only be used for traversal.
   long Position {
     get;
     set;
   }

   // Reads and returns the next line, null if EOF
   string ReadLine ();

   // Reads the next line and returns a stringbuilder containing the line
   // The StringBuilder returned could be the same one used while reading,
   // so it should not be modified and its content might change when readline
   // is next called.
   // This is the most worst horriblest API I ever designed, for sake of speed
   // And thats why this should not be a public API.
   StringBuilder ReadLineAsStringBuilder ();

   // Skips the next line, return true if successful
   bool SkipLine ();

   // Skips required number of lines; returns actual number of lines skipped
   long SkipLines (long n);

   // Close the reader
   void Close ();
 }
}

Beagle source contains the interface and several implementations.

Fasten your seatbelts; we are ready to ship

There was a discussion going on in beagle mailing list sometime ago where I made a comment that I dont think beagle is newbie ready i.e. plug-and-play yet. Beagle fans did not understand my comment and people replied why they think I am wrong. I would be glad to be proved wrong; but all of their arguments were how they were using beagle since version x.y, how beagle was shipped and enabled by default since last z releases of some distribution and how someone is able to install beagle in a large enterprise. Duh! None of these prove that beagle is newbie ready. All they do is show that beagle works and even I know that. 0/100.

I have the feeling that some of the beagle devs and followers live in the garden of Eden surrounded by a high wall of reality. Sometimes they should go out in the streets, check the bugzilla of other distros, go through user blogs (which mostly contain complaints about how beagle does not work and how to disable it), and visit some user forums where a lot of questions are how to disable beagle from starting at startup. These are laborious jobs and not pleasing. A lot of them contain flames and invalid reasons. But almost always they are started by someone who found beagle causing trouble.

Here are some links which can make your task easy:

• Ubuntu beagle bugs
• OpenSUSE beagle bugs
• Fedora Core beagle bugs
• update Gentoo beagle bugs
  

I sometimes make the rounds and all I see are I make a point of uninstalling beagle on all my machines and The first thing I did after ... was to uninstall beagle and now my machine is happy. Silly men, how can they not like the doggy!

Subversion arrives. Finally!

KDE uses SVN for their source code management. GNOME used to use CVS till yesterday, which means beagle too was managed with CVS. I do not know the technical details. but time a again we did face technical problems with CVS. I was mostly told that life would be easier with SVN and lo behold! GNOME has switched to Subversion as its SCM (actually, still switching).

The last time this was tried by the awesome GNOME guys, they later found a glitch and had to cancel the migration. As a result I lost a commit that I made within hours of SVN migration. This time I will play safe and watch it for a few days before committing anything. If everything works out, life should be easier. Joe already cleaned up quite a bit of the unused files and directories, renamed the Evo-mail backend correctly and updated the links et al. A New Year with a clean, new repo. Sweet.

PS: There is one downside though. Joe (and others too) would like to use the SVN commit messages for creating Changelog files during creating a tarball. Which basically means others cannot observe the Changelog file between releases to figure out what was changed (neither I nor Joe updated the Changelog while committing, so this is a lame excuse). The real trouble is now I cannot write any lame jokes in my commit messages. Life would be serious now. Boo hoo.

My time with the doggie

Today I read about the Ohloh project (http://ohloh.net) and added my favourite project beagle (http://beagle-project.org) to it. I was curious what it actually does.

It took them nearly 4 hours to download and analyze the source code. But it was worth the wait. It showed some interesting statistics, like 122,885 LOC codebase, 82 direct contributors (committing in CVS) and 13 of them in last 12 months.

Just for a light comparison, Firefox has a codebase of 157,207 LOC, Amarok has 169,288 LOC and (take this) PHP 6.0 has 599,805 LOC.

It was also amusing to see my share in the project: http://ohloh.net/projects/3826/contributors/21154

beagle 0.2.14

Joe announced the release of beagle 0.1.4 few hours ago. Its a fascinating and shining new release containing exciting new features, lots of memory/speed optimizations and many bug-fixes as well. Here are the major ones which are readily visible:

• Indexes tar, gzipped-tar, bzipped-tar, gzipped and bzipped files, in the filesystem as well as in email attachments. The results show you the exact file in the archive that matched the query.
  
• Do some smart tokenizing to allow matching 001234 to a query of 1234 and better matching of file names. No more missing files.
• Beagle can find and extract data itself using its dozen or more backends. But sometimes its better for other applications to send data to beagle for indexing. Beagle had the infrastructure to act as a search/indexing service provider. The release contains an example C code to show how to do that; its pretty simple actually. Obviously python can also be used.
• Some cool signal mechanism which help to figure out what file in being currently indexed and for how long. This will be helpful if you feel beagle is taking ages to index some file.
• Use Xdg autostart mechanism to auto-start beagle. KDE4 will also implement xdg autostart mechanism. One more step towards being DE agonistic.
• The indexing information now explicitly mentions if the initial indexing is in progress. Also clients now have the option of being notified when the initial indexing ends.
• Lots of memory fixes. bhale just mentioned in the irc channel holy crap, startup RSS for beagled is 15m... beagled is below nautilus in mem usage...  im not believing my eyes :) Thank you for your myth on how beagle is a bloatware.
• API and beage-search support to know the total number of documents that matched any query. Not the superficial imposed limit of 100 documents.

Go, get it!

License to hack-debug-release

I start with the disclaimer that I am not a lawyer. I am also not very careful in reading the EULAs (End User License Agreement - a short acronym of a long term usually describing an even longer gibberish english text). When I release my first piece code (was it JBabel or mGet ?), I put them in GPL (possibly v1). Of course I didnot know what was I doing - I was merely completing a formality during the fascinating experience of sharing one's program. With time, and during development of Beagle, I came to know about the various licenses. It was about this time that the GPLv3 debate started. Also, I started releasing verious kde-beagle softwares which required me to figure out the correct licenses for them.

Liceneses, I feel, serve two purpose, demand credit where its due and specify the amount of responsibility.

Authors of some projects just want to release code without claiming any credit. I did so too. Only to be pointed out by someone that I have to actually specify the terms for others to use my code. Since I dont want to be credit for the code, its natural not to held responsible for its damages either. That needs to be specified too. One can either cook up one's own license or just pick one from the sea of licenses (http://www.opensource.org/licenses/) . The license I found to do just that is the MIT license.

MIT (X11) License: Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, ...

The license is itself not copyrighted, so it can be modified if needed. A rather free-to-do license, except for this clause following the above text

... subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

Wikipedia tells me that the latter part is necessary for the copyright laws of US and other countries. The credit part is followed by the author responsibility part

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT...

Often, people use this clause to explicitly deny responsibility

Except as contained in this notice, the name(s) of the above copyright holders shall not be used in advertising or otherwise to promote the sale, use or other dealings in this Software without prior written authorization.

I havent seen projects where the last clause wasnt present, so I assume it is not a necessary clause. I didnot use it myself when I released yaBi under MIT license. The core parts of Beagle and Mono are released under this license.

My other project, kBeagleBar uses the fancy LGPLv2. I had to use LGPLv2 because I was using some source code from kde libraries. Note that, LGPL allowed me to use the library without any restriction, but I copied some code from a library source file - so I had to use LGPL. Similarly, I release kontrol as (distributed as a part of kbeaglebar) which uses the code from GPLed kerry, so its under GPL too. Also GPL and LGPL are copyrighted by FSF. But then again, I wouldnt dare to modify them myself.

I am looking forward to GPLv3. No, not because I have any particular reason to like it. Because I hope to read it fully, understand it and read the various articles and blogs about it to understand the issues of free software licensing. Its always good to know where my rights end and start.

Even after all this, it will take me some more years to actually read EULAs. They are frightfully convoluted to confuse my little grey cells.

Bigger than life icon

Icons are small pieces of art I encounter in my everyday life. They are like
apparels, you see them in all forms in every moment of life. They serve
different purpose, keep you warm, keep you cool, keep you comfortable
whatever but all them try to look good (some even make it their prime
agenda). In human-computer interaction, icons are large, easy targets that
serve as better visual cues than textual messages that needs to be parsed and
pattern matched. A good icon make that experience fun and enjoying.

I started with the the usual KDE crystal icon theme, and then tried different
icon sets. The tango icon theme is good, but like the usual gnome colours,
they have a very low-saturated colours which look a bit drab and boring on my
desktop. I am not looking forward to extremely gorgeous, highly saturated and
colourful icons; my desktop is not a colour palette, but the art forms I will
stare at a million times over months better be pleasant to look at. kde-looks
has quite a few icon theme, and a lot of them are mixed icon theme. For a
while, I tried KDE Crystal Diamond icons. It has good icons, except some of
the icons are very light coloured, white based and its hard to make out the
icon contents. Most of the filetype icons and the media-play (amarok)
play/pause icons are like that.

I have heard a lot about Oxygen icons; unfortunately their license does not
allow anyone to package them. Of course, I could download the svg icons
directly from their svn and convert them to an icon theme locally. I need to
install inkscape for that. Interesting enough, I found a forum post in
sabayonlinux about using oxygen icons in current kde desktop. The user gives
out the details of how to make an icon theme and later even provides a tar.gz
package to install oxygen icons. Oxygen is work in progress, so that user
replaced the missing icons by other icons. I think he violated their license
but I have to double check. Anyway, I installed that icon them and its really
good. Attention has been paid to every detail. Icons are 3d and 2d exactly
according to their need, those that look good with shadows have them, the
colours look just ok. All in all, I am pretty satisfied. I am looking forward
to manually checking out the oxygen icons and making a theme out of it. That
way, I will get the icons from oxygen as they are checked in. Good work,
guys.

Mono.FUSE Filesystem for Digikam Tags

FUSE is cool anyway. And writing FUSE filesystems in C# is even cooler. Adjectives aside, I wrote a FUSE-DigikamFS filesystem for browsing Digikam tags. Its much easier to browse the images organized by tags using konqueror or kuickshow. This way I can also share the tags mount-point by kpf (or some other HTTP server) allowing others to browse my images arranged by both folders (Albums) and tags.

A sample session,

[debajyoti@dbera Tags]$ pwd
/home/debajyoti/Tags
[debajyoti@dbera Tags]$ ls
Artistic/  Favourites/  Fireworks/  Food/
Nature/  People/  Places/  QuickCheck/  Season/
[debajyoti@dbera Tags]$ ls Artistic/
Black-White/  Mood/  Motion/  pa020099.jpg@
pa020102.jpg@  Perspective/  Shadow/

As you see, I use symlinks to overlay the images from tag filesystem. Works pretty cool.