The Darxite FAQ

What is it?

"Darxite" is the name given to a system consisting of a daemon running in the background whose job is to retrieve or upload files via FTP or HTTP and execute other FTP commands, and a number of "client" programs whose jobs are to control the daemon and to provide a user interface. Darxite is the ideal program to use when you need to get some files over an unreliable network connection.

What are the requirements to use it?

Darxite has only been tested on Linux kernels 2.0 and above, and with glibc2. It may work on other Unices and with libc5 - try it, and tell me if it does or doesn't. The daemon does not require any other libraries, but libpthread is optional and improves performance.

Unsurprisingly, the GTK client programs require GTK (1.0+ should do) and the GNOME clients require GNOME 1.0. The command-line FTP client requires a reasonably recent version of GNU readline.

Why was it written?

Darxite was born from the concept of having a process sit in the background and get files for you quietly. As a dialup user in Britain, with its overly high phone charges during the week, if you want to get a large file, it's not really worth loading up an FTP client and going through all the hassle of finding the file when you'll only have time to download a tiny bit of it. With Darxite, you can make progress in downloading it even when you're only connected for a short time.

Plus, Netscape crashed one time too many. If the server supports file resume, Darxite will never delete partially-complete files - unless you tell it to.

What nice features does it have?

What are the differences between Darxite and GetRight?

Firstly, GetRight only works under Windows 95/NT, and it's not open source. But apart from this, it's based around a different model: it is simply intended to help the user download files using Netscape, Internet Explorer, and so on, and does not have a "client-server" system where applications can control it. This makes it more inflexible; for example, you are forced to use the provided GUI. Darxite also has some features that GetRight does not; for example, remote control. As of version 0.4, Darxite includes a script for converting your GetRight bookmarks.

Why not use another Unix utility, eg. ncftp, sftp...

Darxite has a number of nice features, such as the ability to switch to a mirror site if transfer rates get too low. It was designed from the outset for multiple simultaneous transfers and for batch operation, and can be easily scripted by use of the various helper utilities such as "darxget" and "darxcmd". You can use a command-line or X-based GUI, and switch between them at will without losing data - eg. if X crashes, you'll still be OK. The spooling system means that you know when a file transfer has completed. And finally, its name isn't an acronym involving the letters "ftp" :)

Why didn't you base it on an FTP library or something?

None of them seemed to support features I needed, like multithreading. Also, I wanted to really understand our FTP/HTTP backend and to get the best performance out of it we could. We didn't just script an FTP client (such as curl) because that would have been very limiting.

Why is it called "Darxite"?

For the following reasons and many more:

Why the odd release names?

If Microsoft can name their releases after American towns, and Debian can name theirs after characters from Toy Story, why can't we name ours after characters from an amazing set of books? To be specific, the Hitch-Hiker's Guide to the Galaxy quintet. But if you've never heard of them, maybe you aren't the sort of person we want using our system anyway :)

So how do I simply download a file from the command line?

Use the provided client program "darxget". For example, you might type: 
darxget ftp://ftp.microsoft.com/windows/98/full-source.tgz
...if it existed. There are many command-line options to select things like the output filename: type "darxget -h" to see them. An especially useful one is -b for blocking mode; ie. the utility will wait for the file to complete its transfer before continuing.

Is there any snazzier way to transfer files?

Lots. Try the command-line FTP client dxftp or the GUI client NoctFTP. Or the GNOME Panel applet. But you should really read the USAGE file; it should tell you everything you need to know.

What clients are included as standard?

The following clients are included in the standard Darxite tree:

How do I pause or cancel files from the command line?

Type 'darxcmd "pause [url]"' to pause the file with [url], and 'darxcmd "continue [url]"' to resume transfer. To cancel a file, type 'darxcmd "cancel [url]"'. If you want to pause, cancel or resume all the files in the batch, simply type 'darxcmd "pause all"', etc. You may like to alias these commands, eg. to "dxpause", "dxresume" and "dxcancel", if you use them a lot. Note that the quotes are required!

What's all this about controlling the daemon remotely?

You can run the Darxite daemon on a remote computer and control it via the network. You can even use clients like NoctFTP, which can be really handy if the remote computer is slow or doesn't have X, or whatever. Read the USAGE file to find out more.

Is there a Windows client I can use?

Not yet. But what you can do is use telnet, which works almost the same as darxcmd does. See the USAGE file, under the section "Telnetting to Darxite", to find out more. This is kind of clunky and user-unfriendly, but it works.

Does Darxite support FXP?

No. Transfer from one FTP server directly to another (so-called FXP) is not supported, and probably never will be, for the following reasons. Many modern servers do not allow it because it is a security risk, and Darxite is intended to be a utility for transferring files to a local computer, thus enabling many features. Also, if you have login access to one of the servers, you can achieve much the same thing with Darxite by running the daemon remotely on either server.

Can I tell Darxite to download a file at a certain time?

Not directly, but you can use the "at" command to do so, eg: "at 9.00am darxget ftp://some.server/some.file". And you can even get Darxite to e-mail you when it's finished, or to automatically disconnect from the Internet. If you want it to be downloaded at the same time every day, you can use "crond" to do it. If you set up your computer nicely, it should even be possible to tell it to dial up, download the file and then disconnect...

Darxite uses a huge amount of CPU time. This is evil!

If there is a very large data transfer rate (eg. when using FTP locally), Darxite will use a large amount of CPU time. If you don't like this, run it via nice, eg. "nice -n 19 darxite" would give the program a very low priority and make it use less CPU time.

I've downloaded a file but I can't find it. Huh?

By default, all files are downloaded to your home directory, and transfers are spooled - ie. files are saved in the directory "$HOME/.darxite/spool/" and are only copied when they are complete. If the transfer is canceled, Darxite will not delete the partially-complete file, so it'll stay in the spool directory. You can change all this behaviour via your ".darxiterc" file. For example, to download all files to the current directory, add the line "OutputDir ." to the file.

ps shows I've got two copies of Darxite running. Why?

The way that threads work in Linux means that you'll see an extra line "darxite" in ps' display for every thread running - plus one for the thread scheduler. It shouldn't actually be possible to run Darxite twice as the same user, unless things have gone very wrong.

I want to write a Darxite client. How easy is it?

Very easy, especially if you use the Darxite library, which essentially makes it a matter of calling one function to connect to the daemon, one to send commands, and one to disconnect. If you don't want to use the library or aren't using C, it's still easy and requires very few lines of code. Take a look at the file "develop.txt" and e-mail Ashley for more information.

Are there any clients that you'd really like to be written?

Yes. A few of the ideas we've had are: an ncurses client, a QT client, a KDE Panel or Afterstep dock applet, a Perl client (especially useful for Web sites), a TCL client (erm...), an Xlib client (even more erm...) and so on. E-mail me if you come up with any ideas.

Are there any performance issues with running loads of clients at once?

Clients talk to the daemon using multiple sockets and asynchronous I/O and only when they need to, so there is no performance impact entailed simply in running lots of clients at once (except that a small amount of extra memory is allocated for each client), but running many clients that poll the daemon repeatedly - such as darxstat-GTK - will cause a performance hit, though it won't really be noticeable with most transfers from remote sites.

I use KDE. Is there a panel applet like the one for GNOME?

No, there isn't (yet). But there is a simple way you can get some of the functionality of the GNOME applet. Just create an "Application" link on the panel (or even on your desktop) and set the command to execute to be "darxget %u". Then, if you drop a URL on the icon from KFM or any other KDE client, it will silently be retrieved in the background. This won't work with Netscape links, but maybe in KDE 2.0 it will.

I found a bug in Darxite...

Doh! Make sure you tell me about it, if it's repeatable. If it's in the daemon, enable error logging (via dxpref or editing your .darxiterc), and e-mail me the "log-errors.txt" file produced. If not, just tell me the problem and I'll try to fix it. I'm also very happy to receive feature requests, no matter how insane.

I don't like Darxite. Does it have any competitors?

If you want a utility to get URLs from the command-line, I recommend curl, wget and got_it. If you want a command-line FTP client, why not look at ncFTP, sftp or lftp. As GTK FTP clients go, gftp and IglooFTP are worth trying. Many ideas have been liberally stolen from all of these. If you would like a client you can drag and drop a URL on, GTransferManager (for GNOME apps and Netscape) and Caitoo (for KDE apps) might be worth looking at. And if you have to use Windows, there's GetRight and Go!Zilla, which are unfortunately non-free.

What sort of features are you planning to add in the future?

Lots.
Ashley Montanaro
Last modified: Wed Nov 9 12:30:41 GMT 1999