Making a fool of your applications - File System

An understanding of how your applications work will make it a bit easy for you to fix problems when they arise. One common scenario I have seen in most applications is the concept of application logging - creating logs using either the old-fashioned log files or storing the data in a database. Logging helps application developers troubleshoot application-related problems in production environments when the need arise. I've had my shares of creating application logs in the past where I use text files to store activities happening in the application. There's a downside in using this concept. One, you are introducing additional overhead in your application since you have to do an additional step, not to mention the I/O necessary to write the log in the file system or the database. Another downside is log maintenance. If you do not maintain the logs, it wouldn't take long before they fill up your disk, causing your application to fail. There are ways to work around this. You can toggle logging and only turn it on when needed. Another approach is to create a default log maintenance procedure in the application itself like truncating logs or deleting log files older than a specific date. Maintenance will be a terrible headache if not included as a part of the logging mechanism.

I have had the opportunity to deal with such a case. I had an encounter with an application which logs every transaction by creating XML files. XML is a great way to store data. But what I have seen for the past few years is that the use of XML has been misunderstood as something to replace a relational database. This misunderstanding of its purpose has caused a lot of problems particularly when it comes to performance. You see, in order for you to read the data in the XML file, you have to load it in memory before you can even do those methods as parsing using XPath and XQuery. Imagine doing this to load a million XML files. My problem was to delete the log files stored as XML. I couldn't just delete them since they have increased in number that simply running Windows Explorer has caused my session to hang. My next step is to delete the folder containing the log files. But it's not as easy as that. The folder containing the logs is being locked by the application which is typical of all applications creating logs. To work around this problem, I had to find out what application or service is, stop it the service, rename the folder containing the logs, create a new folder with the same name as that of the old log folder, restart the service and, then, delete the old folder. The application will still see the logs folder except that now it's a new folder but with the same name. This made sure that I can still do maintenance by deleting the logs folder while making sure that application downtime is kept at a minimum. Bottomline still remains, we people are indeed smarter than these machines.