LnBlog

It looks like LinkSleeve has been failing me lately. The past week I've been inundated with comment spam. So, to fix that problem, I updated the BotBlock plugin. You can download version 0.2.0 here. Just unzip and drop it in your LnBlog/plugins directory, overwriting the old version.

For the new version, I basically rewrote the plugin. The old version did nothing but force bots to download the comment form before submitting, which was fairly useless.

The new version, however, implements an invisible CAPTCHA. This is not the annoying image-based CAPTCHA you see all over, but rather a simple math problem. Furthermore, it's "invisible" to anyone with JavaScript enabled - i.e. it includes JavaScript code to automatically solve the CAPTCHA and hide the form field. So, for 90% of visitors, the comment form should work as before, but block any bots that don't interpret JavaScript (which is a lot of them). For the rest, there will be a simple math problem and they will be prompted to type in the answer. (Note: There is an option to turn the non-JavaScript text off, if you're so inclined. Of course, that makes it less accessible, but apparently lots of people are into discriminating against the disabled.)

As a fall-back measure, I also included an option (on by default) for some simple content filtering - that is, comments with HTML links in them get blocked. The idea is that most of the comment spam I get includes plain text, HTML, and BBCode links, but LnBlog only allows plain text in comments. Therefore, anyone who posts a comment with an HTML link is either a spammer or too stupid to read the instructions. Either way, we probably don't care what they have to say.

So there it is. I haven't had any comment spam since installing the new version on my site. Of course, I'm still having problems with TrackBack spam for some reason. Apparently the LinkSleeve and TrackbackValidator plugins aren't working for this particular wave, though I don't know why. I'll have to work on that.

I've just uploaded a new version of the BotBlock plugin. This one adds two options. One makes the comment check depend on a cookie, so user agents that don't accept them will be blocked. The other option blocks any comment that contains markup for HTML or BBcod-style links. Since markup isn't allowed in comments anyway, it's a pretty good bet that comments with links are spam. Either that, or the commenter wasn't paying attention.

I've been reworking the sidebar calendar plugin lately. It was kind of buggy and just didn't work as well as I would have hoped. In particular, using query strings to browse through the months just really sucked.

My solution to the bugginess was to rewrite a large chunk of it. It's much cleaner and more managable now.

My solution to query strings sucking was to get with this whole Web 2.0 thing and do it with AJAX. So now, rather than the whole page refreshing when you click the previous month link on the calendar, a JavaScript requests an update from the server and then updates the page in place. The down side of this is that the next and previous links don't work if you don't have JavaScript or are using an archaic browser. Of course, you can still use the archive links, so it's not like you can't get to the previous month's links.

I've installed the new calendar plugin on this site. I haven't made it available for download yet because it's still in progress and requires a couple of support files. However, it will be standard in the next version.

Following up on the post I just made about the BotBlock plugin, I thought I should mention that the reason I'm getting comment spam is that I turned off the DisableComments plugin on LinLog. The reason I turned it off was to test out the TrackbackValidator plugin that comes standard with LnBlog 0.8.0.

The DisableComments plugin allows you to automatically turn off replies (TrackBacks and comments) on entries older than a given number of days. Since my big problem was TrackBack spam, and it was mostly on entries that were more than a month old, I "fixed" the problem by simply setting DisableComments to disable replies after 30 days. On the up side, this stopped the flood of TrackBack spam. On the down side, it stopped all legitimate replies too.

Well, it turns out that I don't really need the DisableComments plugin as much anymore, Happily, the new TrackbackValidator plugin, which only allows TrackBacks from URLs that actually link to you, has completely solved my comment spam problem. My server access logs will show lots of TrackBack pings, but not a single spam ping has gotten through.

The only down side is that now I need to worry about comment spam on old entries.

I added a new plugin to the plugins page today. It's called BotBlock, and it's just simple attempt to keep robots from posting comments.

I wrote this because, for the past several days, I've been getting a lot of comment spam. The messages were comming in groups of two to six messages at a time, had varying content, and came from varying IP addresses. However, the general format of all the posts was the same (short fake greeting, followed by lines of URLs and two or three word descriptions) and they all targeted the same blog entry. So obviously these were either being posted by a robot or a very stupid human.

Thus I implemented this stop-gap solution. Basically, it just adds a hidden field to the comment form that contains a hash value based on your LnBlog configuration and the client IP address. When the client submits a comment, it checks this hash. If it's either missing or doesn't match the calculated value, the comment is rejected.

Of course, this depends on the bot being relatively stupid and the spammer not being motivated enough to figure out your specific configuration. A determined spammer could bypass this protection without too much effort, which is why I call this a stop-gap. However for small-time blogs like mine, which aren't worth the effort to crack, this solves the immediate problem.

I read an interesting article on TrackBack spam today, called Taking TrackBack Back (from Spam). The ideas seemed sensible to me, so I banged out a TrackbackValidator plugin to implement it.

The idea behind the paper and the plugin is simple: the sites linked to by TrackBack spam never contain a link back to your blog entry, but nearly every legitimate blog entry that pings you does. Therefore, rather than fancy content filter, we can eliminate TrackBack spam by simply fetching the URL in the ping and only accepting it if the page actually contains a link to your site.

This plugin implements that idea. It also includes options to white-list all pings coming from your domain and to allow ping that link to files under your entry (e.g. an uploaded file or the comments page) rather than just the permalink. My spam rates have been way up lately, so I'll be uploading this tonight and we'll see how it goes.

I've uploaded a new plugin to the plugins page. It's called HideSidebar and it, well, hides the sidebar. The idea came from this comment and the plugin is a sort of proof of concept. I don't know how useful it will be, but it's a good demonstration of the power of the plugin system.

For the curious, it works by injecting an inline JavaScript into the page when it loads. It has PHP code to add a link to the menubar and insert the JavaScript into the page. The JavaScript then manipulates the DOM to hide the sidebar when the link is clicked. It's really quite simple. In fact, the JavaScript portion took me longer to write than the rest of the plugin.

I've added a couple of new plugins to the plugins page.

The RecentComments plugin is a sidebar panel that shows a list of the most recently posted plugins. It's got configuration options to control the display of the poster's name and the entry it's posted to.

The new verions of PrivateBlog, which used to be part of the standard set, lets you block access to a blog by all except a given list of logged-in users. You can allow access by username or by group membership. In addition to redirecting pages, the new version also blanks out all entries, articles, and comments that might be displayed by other plugins (like in the sidebar). However, this only works for dynamic PHP pages: static files like RSS feeds are not blocked. So if you really want the blog to be truly private, you might want to consider turning off the RSS plugins.

Also, I fixed some broken and missing links on the plugins page. The two plugins I added yesterday now have links associated with them and the links to the two editor plugins are fixed now.

Ask and ye shall receive! By popular demand, I cooked up a little contact form plugin today. You can download it from the plugins page. To install, just extract it from the ZIP archive and drop it in your LnBlog/userdata/plugins folder. (Note: This is the new recommended location for non-standard plugins. You can still use LnBlog/plugins or your per-blog plugins folder as well.)

Basically, this plugin provides a contact page that users can use to send e-mail to the blog owner. It also puts a link to the contact page in the sidebar. Note that the blog owner must have an e-mail address set.

This is just the initial release, so I'll expand and integrate this more later. You know, putting the link in user profiles, allowing more than just the blog owner to be e-mailed, etc. I'm trying to go with that whole open-source "releaswe early, release often" thing here. Plus this is only the second plugin I've done that generates and entire page, so I wanted to work on the basic design before worrying too much about features.

It's time for part 2 of the LnBlog themes tutorial series. In this episode, we'll cover modifying your sidebar. This will include a brief discussion of how theme templates work as well as an introduction to the plugin system. In the process, we will also create a very simple and extremely unimpressive plugin. Note that in this tutorial, unlike the last one, I will assume that you have a basic knowledge of HTML and CSS. This tutorial will also include some PHP code, although I will nto assume a working knowledge of PHP (although if you have one, it will help).

As you may recall from last time, LnBlog's theme system has a concept of paths. This applies not only to images and style sheets, as we saw before, but also to template. In other words, LnBlog will look for a given template first in your blog's templates directory, second in the templates directory of your current theme, and lastly in the templates for the default theme. This means that you can easily modify the templates for each of your blogs individually or for all of them at a time.

Modifying the template

Let's start by making a copy of your LnBlog/themes/default/templates/include_sidebar.php file and saving it on your local hard drive so we can play with it. If you open up the file in a text editor, you'll notice that there's almost nothing in it. It should just look like this: activateEventFull($tmp=false, "sidebar", "OnOutput"); $EVENT_REGISTER->activateEventFull($tmp=false, "sidebar", "OutputComplete"); ?> Of course, if you don't know PHP, that's probably doesn't mean anything to you. In fact, it probably doesn't mean much even if you do know PHP. That's because this code is part of LnBlog's event system

As you may already know, the default page banner, menubar, and sidebar in LnBlog are all implemented as plugins. If you open up your include_banner.php or include_menubar.php, they'll contain similar code. Basically, this code raises an event, i.e. it tells LnBlog's event manager that something interesting is happening and any plugin that's interested in this event better get it's act together. The event manager them checks a list of plugins that signed up to be notified when this event happens and tells them to do their thing.

Now, since this is a template, we can actually ad HTML code right into it. For illustrative purposes, I'll add some of my favorite links to the sidebar. Here's what the resulting code looks like: <?php global $EVENT_REGISTER; $EVENT_REGISTER->activateEventFull($tmp=false, "sidebar", "OnOutput"); ?> <h3>Recommended Links</h3> <ul> <li><a href="http://blogs.msdn.com/oldnewthing/">The Old New Thing</a></li> <li><a href="http://www.securityfocus.com/">Security Focus</a></li> <li><a href="http://www.larkware.com/">The Daily Grind</a></li> </ul> <?php $EVENT_REGISTER->activateEventFull($tmp=false, "sidebar", "OutputComplete"); ?> Notice that I broke the PHP code into two blocks and put the HTML in between them. Since tempaltes are actually PHP files, anything inside the <?php ?> tags is treated as PHP code, and anything outside them is treated as HTML. If you're sharp, you probably noticed that I did this because of the event code: I put my markup between the OnOutput and OutputComplete events.

If you save this file and upload it to your blog's templates directory, you'll see your list of links in the sidebar. However, you'll notice that the links are at the bottom of the sidebar. That's because all the plugins are loaded by the OnOutput event. If you move the HTML code above that event, then all your links will end up at the top of the sidebar. But what if you want your links in the middle somewhere? Maybe you want your links between your articles and your RSS feeds. Well, that's why this isn't the recommended way to add to the sidebar.

A simple sidebar plugin

However, there is good news. Writing a simple plugin to display some links in the sidebar is easy. Really easy. In fact, it's about a dozen lines of boiler-plate PHP code with your HTML inserted in the middle. And once you have the plugin written, it will be detected by the plugin manager and you will be able to change where it appears in the sidebar by changing its load order in the plugin loading configuration page.

Here's a plugin version of the links in the example above: <?php class MyLinks extends Plugin { function MyLinks() { $this->plugin_desc = "Shows my favorite links in the sidebar"; $this->plugin_version = "0.1.0"; } function show_links() { ?> <h3>Recommended Links</h3> <ul> <li><a href="http://blogs.msdn.com/oldnewthing/">The Old New Thing</a></li> <li><a href="http://www.securityfocus.com/">Security Focus</a></li> <li><a href="http://www.larkware.com/">The Daily Grind</a></li> </ul> <?php } } $plug = new MyLinks(); $plug->registerEventHandler("sidebar", "OnOutput", "show_links"); ?> This is about the simplest plugin you can have. It is a simple PHP class with a constructor that defines the version number and a short description, and a single method that dumps some HTML output to the screen. Note at the bottom that you have to create an instance of the class and register the method with the event manager.

Now is probably a good time to mention that plugins use a path mechanism too. In other words, you can have plugins that apply only to a single blog, just like you can with theme template and style sheet. Just make a "plugins" directory in the blog's directory and put the plugins there. They will be treated just like a regular plugin, but no other blog will be able to see them.

So, now let's install your new plugin. Copy the above code and paste it into a new file named sidebar_mylinks.php. Note that it is very important that you not have any blank lines or spaces outside the PHP <?php >> tags, as this will cause error messages due to the way PHP handles output. Now, create that plugins directory in you blog's directory on the server and upload the sidebar_mylinks.php file into it. If you open up your blog in a web browser, you should see the new links. If you change the load order of your sidebar_mylinks plugin from the plugin loading page, then the link section will move in the sidebar.

If you want to have several independant sidebar sections, you can make copies of this plugin to achieve that. Just change the HTML code, the plugin_desc on the fourth line, and change all three instances of the name MyLinks to MyOtherLinks, or something like that (the exact name doesn't matter, so long as no two plugins have the same name). If you're feeling adventurous, you can also adapt this to add markup to the banner or menubar by changing the "sidebar" parameter in the registerEventHandler line to "banner" or "menubar".

You can download files for this tutorial here. In the next installment, we'll go into theming the content areas of a page, including templates for blog entries. We'll also cover modifying the associated style sheets and possibly adding some images.

Just as a change of pace, I'm releasing a new version of a standard plugin. Actually, I'm doing it because the comment and trackback spam is starting to bug me (it's been picking up again lately), so I figured I'd adjust one of the plugins to get rid of it. And since I'm not planning a new release any time soon (unless I get some bug reports), I figured I'd just release it separately.

You can download the DisableComments plugin version 0.2.0 here. To install, just extract the PHP file from the ZIP archive and copy it into your LnBlog/plugins folder, overwriting the old version. This version adds an option to automatically disable comments and trackbacks on entries older than a certain number of days. To enable the new feature, just go the the plugin's configuration page and enter a number in the text box. That'll stop the trackback spam on entries that are a year old!

Alert LnBlog uses who look at the sidebar may notice that the search section has changed. I've finally gotten around to writing a native search plugin. Yes, I finally got sick of waiting for Google to spider my site. It's pretty simple right now; it just searches for entries matching all search terms. It will also accept a regular expression if you enclose it in forward slashes. This will replace the Google search plugin as the standard search feature in the next release. The Google search plugin will still be included for anyone who wants it, but will be disabled in teh default configuration.

There is one interesting thing about the new plugin: it's the first plugin I've done that displays an entire page on its own. By that I mean that the sidebar panel and the search results page are both displayed by the plugin file, rather than using a separate page file for the search results. I still have a few bugs to work out of it, but if and when I get it sorted out, this plugin will provide a nice template for future self-contained plugins.

Well, it looks like the new IP blacklisting plugin has solved my trackback spam problem. All the spam was coming from two or three subnets, so I just banned the entire subnet and I haven't had anything since.

In other news, I implemented a couple of fixes and features today. First, I fixed a problem with links to uploaded files being broken by the new pretty link feature. Second, I fixed that annoying problem where tags weren't preserved in the edit box when previewing an entry.

On the feature front, I added the ability to turn off trackbacks. For now, there's no seperate setting for it - it just follows the setting for comments. I also added a plugin that allows you to turn off trackbacks or comments for an entire blog. That should be nice for people like me who never actually get any legitimate trackbacks.

Anyway, I want to do some testing, but I think I'm pretty much done with this version in terms of features. Look for a new release before the end of the week.

Well, it looks like testing out the IP banning plugin for a few days was a good idea, because it didn't work quite as well as I'd have hoped. In other words, I'm still getting trackback spam.

I took an hour or so to rework the plugin. This time I decided to keep it nice and simple. I made sure to remove all extraneous whitespace, separate the per-blog and global IP lists, and did the check with a simple preg_match() call. This has the added benefit that I can now use the same code to ban an entire subnet just by including a star in the IP.

I think I should also include a setting to turn off trackbacks and/or comments altogether. I'll bet there are a lot of people who just don't feel like trying to keep up with the spammers. And at this point, I really can't say I blame them. My site isn't even that popular, and I've gotten nearly 50 trackback spams just since I implemented e-mail notification the other day.

Well, now my spam problem is (partially) taken care of. Today I added trackbacks to the IP blacklisting plugin, added a plugin to send e-mail notifications for new trackback pings, and added an interface to delete trackback pings. After I burn them in for a few days, I'll put up a new release.

I would have liked to just release the plugins, but it turns out that isn't possible. You see, I haven't actually touched the trackback code since I first added it. As a result, the trackback class wasn't raising any events for plugins to hook into. In fact, the class didn't even have a delete method. So I had to add all of that. The good news is that now it's taken care of, so hopefully in the future I can just release plugins.

Sigh.... And here I had just finished an IP banning plugin for comment spam and I discover that I'm now getting trackback spam. Pornographic trackback spam, no less. And I didn't even bother to implement an interface for deleting trackbacks, much less blocking them. I guess I'll be doing that tomorrow. My apologies for the oversight.

So it looks like I've got another couple of plugins to implement before the next release. For one, I'll want e-mail notifications of trackbacks, so that I'll actually know when I get trackback spam. Second, I'll have to extend the comment banning to trackbacks. Third, I guess I'll have to start working on that keyword banning idea I was thinking of for comment spam.

If you're exceptionally alert, you might have noticed that LnBlog has acquired a new feature: tags. Yes, you can now categorize your posts by topic. I've even added a sidebar plugin to allow readers to pull up all posts with a given tag. You can also view them as a single page, kind of like a sub-blog.

For anyone not familiar with the term "tag," tags are one of the hot things in "social software." They're basically just free-form categories. Rather than have some pre-defined list or hierarchy of categories, you just pick your category on the fly. If it's never been used before and is never used again, that's not a problem. If several categories seem appropriate, that's no problem either - just tag your item with all of them. It's a great way to communicate associations that might not fit any strict organizational model, but which could still be interesting to others.

However, that's not the reason I chose to add tags to LnBlog. The reason is that it was simply easy to implement. I figured it would be nice to have some sort of categorization system, but I didn't feel like writing an interface to add new categories. So I decided to just implement tags instead and do everything on the fly.

Working with tags in LnBlog is quite simple. When you create or edit an entry, there will be a text box to enter tags. You can enter anything you like in this box. If you want to tag the entry with multiple terms, just separate them with a comma. (Note: This can be changed with the TAG_SEPARATOR configuration constant.) So, for example, putting in "Linux,KDE,FreeDesktop.org" would tag an entry as being related to Linux, to KDE, and to FreeDesktop.org. When you search for any one of those tags, that post will show up. In addition, the blog itself will keep track of any tags you use and expose that list of tags for use by plugins. That's how the sidebar topics plugin works.

I'll probably have the next version with this feature up some time before Christmas. But first, I want to add another feature: a plugin loading configuration page. Basically, this will allow you to configure which plugins are loaded and set certain plugins to load first without having to mess around with moving or renaming the files. Currently, the interface for it isn't too pretty, but it's better than nothing. I'll polish it up in a future release.