BookMooch Blog

I’m experimenting with some changes on BookMooch, in order to deal with some of the growth the site has experienced.

That means BookMooch might be a bit slow for the next 48 hours, while I try a few experiments, enabling and disabling various technologies.

Update on Tuesday, June 3rd: I’m done with my tests, and BookMooch should be fast again now.

An aside: I received an email from the New York Daily News today, and they asked me how many mooches there are per month.

I wrote her:
In the past 30 days, 65,263 books have been swapped. We’ve grown a lot recently: on January 1st, the past 30 days yielded 38,948 swaps, so that we’ve grown 67% in the past 6 months.

---

Notes for techies…

The rest of this blog entry is only interesting to sysadmins and programmers, for those interested in the tech details of what I’m up to…

sorting

Pretty much every page at BookMooch needs to alphabetically sort a list of books. Sometimes, like with an inventory, it’s a small list, but other times, such as a search for “science” or look at a topic like “fiction”, there are a half-million books to sort.

I have to use a custom sort algorithm, because people want books sorted by “Author’s last name, Author’s first name, then book title”, and it can be tricky to figure out what an author’s last name is. Hence, a custom sort.

To speed up the sorting, I maintain a database table, which for each ISBN, has the pre-built “sort key” for each book (author’s last name, etc…). I then load that up into memory.

Nonetheless, sorting a half-million books just for one user’s page request is pretty time-consuming.

So, what I do is save the sorted results in a disk cache, so that if the same unsorted list of books is wanted, presto! it’s already there, all sorted.

However, if that sorted list changes even by just one book (which it would frequently if it were a search for “science”) then that cached, sorted list of books isn’t of any help.

Also, that’s a lot of writing to the disk, especially if a lot of those cached results aren’t useful.

This “sort cache” gets pretty big, about 10gb in a week’s time. Big enough, that using the cache isn’t nearly as fast after a few weeks as when we started, which is why sundays (when we backup) get slower after a few weeks.

Tomorrow, I will try a relatively small (100mb) in-memory sort cache, to see what the performance characteristics of that are. I’ll use an MRU cache algorithm, and that should speed up common searches and “next/previous” page requests, while not causing disk writes and having to delete old cache entries.

slow drive reads

Web site usage of BookMooch is currently at 400,000 page requests per day, of which 75,000 of those are searches. This is a 50% increase since the new year, and I have no reason to think the growth will stop.

The BookMooch database, at 14gb, is getting a bit large to hold in memory, and a I foresee a time, soon, when most of the database won’t fit in memory. Putting the database in memory is what I’ve been doing thus far to increase performance, and that technique has worked well.

Unfortunately, disk read speeds are pathetic, at around 1200k/second (peaking at 6mb/sec). They’re this slow, despite my using the newest, highest-speed server-class hard drives. I believe they’re this slow because the database mostly does small random-access reads, which are notoriously inefficient.

If I restart BookMooch, I generally pre-load the entire database into memory before letting the public at the site. However, this now takes almost an hour, which is a long time to be down. Today, I just put the site up without preloading the database into memory, so that it runs slowly, but it’s available right away, and the database loads into memory over the course of a few hours.

I’m thinking that a possible solution to my performance problems is to use a solid-state-drive (SSD). The random-access read times are amazingly fast, but random writes are not so fast.

My experiment today was to turn off the on-disk sort cache, in order to lower the number of disk-drive-write operations I need to perform. That’s why BookMooch is running slowly right now. I want to see, however, once the memory cache is filled, how much of a performance hit this causes.

I’m looking at the drives from Mtron and this SSD drive in particular. This blog by the CTO of Spinn3r has many postings about SSDs in server environments, and the results are encouraging.

Over the next few weeks, I’m going to try tweaking some things, and will likely switch to an SSD as the main BookMooch drive.

—

Update the next day: disabling the disk cache caused too big a performance hit. So, I tried using a 1GB in-memory cache instead of the on-disk cache, and that seems to work well, so that’s what I’ll stick with for now. Speed is good, though the server-load is higher than usual, but this may be due to the fact that I’m recalculating “recommended books” for each book as people visit them, whereas previously the recommendations never changed. I won’t know for sure for a few days. The web site feels very responsive, though.

Jon Udell, the host of IT Conversations, interviews me on this week’s show.

http://itc.conversationsnetwork.org/shows/detail3645.html

I’ve long admired Jon Udell’s columns in InfoWorld magazine, and it was really exciting to me to have an interviewer contribute so many insights, so that it really was a conversation, not a Q&A.

The interview goes over a large number of topics, from my company Lyris to Magnatune to WreckAMovie and BookMooch.

-john

In about a week, I will be printing mooch cards in the UK, and having volunteers send them out to people who want them. These are the little cards you can give to your friends when you tell them about BookMooch.

If you live in the UK, and want a box of 1000 mooch cards, please go ahead and mooch a box of 1000 cards now.

By mooching now, you can help me predict how large a print run I should order.

I will only be sending out boxes of 1000 cards each, because I want people to:

* include cards when they send books to others
* make little envelopes of 25 or 50 cards, and offer them to others for mooching

in this way, the cards should make their way around a bit.

The BookMooch journals project continues to blow my mind.

Rena Rossner recently made a new graphic for the project, riffing on the existing illustrations, and I absolutely love it (it’s on the right). It really captures the idea of gluing, writing, drawing, doing whatever you want, to an existing blank book.

Like the BM Angels graphic, it’s a great example of what a creative commons license on artwork can enable (re-use, new ideas, based on existing works).

Alicia Edwards is providing tons of her energy into the project, and it really shows! There’s a dedicated domain for it now, at http://bookmoochjournals.com/ and a huge increase in activity.

A few days ago, Alicia posted links to a whole bunch of new journals, showing many diverse ideas:

* Open the Door to Your Dreams
* Secret Family Recipes
* Cats Do the Darnedest Things
* Saturday Night at the Drive-In
* Into the Blue
* Brush it Off
* Behind the Veiled Curtain
* Bands and Their Devoted Fans
* Found Treasures of Life
* Dice Throws of Life

And be sure to look at this incredible list of the existing journals now in circulation.

many journals are uploading photos, showing the amazing creativity at work:

* Handscapes
* Life’s a Beach
* Open Theme - All of Us
* The Golden Opportunity
* Favorite Books

There is now a Flickr Collection of Journal photos, which is very impressive.

I suggested on the BM Journalers discussion group that I’d be willing to pay to get full color print-on-demand copies of finished journals into circulation on BookMooch, so that the efforts yield a permanent artifact that many others can enjoy. People liked the idea, and that’s what I plan on doing as journals get completed.

Here are some flickr photos of a few I quickly spotted, which will give you an idea of some of the visuals being created:

I’ve posted various metrics about BookMooch before, using numbers from the BookMooch Statistics page. Today, I decided to take all the monthly web logs, combine them into a big excel file, and see what that looks like.

My logs start November 1st, 2007: before then I wasn’t saving the web site log data. On most of these charts, I’ve added a 14-day trend-line, to quiet the noisy day-to-day fluctuations.

---

This chart of daily unique visitors (ie, individual people) shows us at around 18,000 daily visitors, which is 48% more than 6 months ago.

---

This chart of daily web page hits shows a 45% growth in the past 6 months, with 300,000 page hits per day currently.

Both of these charts are understandable when you look at our BookMooch members listing books to give chart, which also shows a 50% growth in the past 6 months.

---

This chart shows when people visit BookMooch. Predictably, it’s a bell curve, with the bulk between 8am and 8pm (Pacific Time). I made a similar chart for day-of-the-week, but I’m not putting it on the blog, because it showed that usage is almost identical for each day, except that Wednesday was about 20% less busy.

---

This chart shows how long people stay on the BookMooch web site. Most interesting to me was how many people stay a fairly long time, with 45% of visitors staying more than 15 minutes. That is quite unusual. Note that I removed from this chart people who stay less than 30 seconds, since they obviously didn’t find BM interesting, and left right away — I was interested in seeing how long everyone who actually uses the site stays.

---

No surprise here, 50% of BM visitors use Internet Explorer, and 37% use Firefox.

---

When I get started making charts, I have trouble stopping!

Here are two more, and then — really! — I’ll get back to work.

I’ve always wondered what our quarterly growth looks like, since that’s how public companies report their numbers.

This chart shows how much more mooching occurs each quarter vs the previous quarter. Yes, I know that we started at high growth rates (that’s why the curve goes downwards from right to left) as that’s always the case with a new company, since you’re starting out with very small numbers, so it’s easy to have growth.

For the past 9 months, we’ve stabilized at around a 27% quarterly growth in mooching of books:

Interestingly, the growth in active members (people who have books listed for giving) is quite a bit lower than the growth in mooching. For the past 11 months, we’ve been around a 12% quarterly growth rate in active members. This shows that BookMooch is become “more useful”, in the sense that the people who are on the site are mooching more and more, at a much higher growth rate than the acquisition of new members:

A nice article on BookMooch today at MoleskineCity.

On monday I had dinner with Xavier Helgesen, one of the cofounders of Better World Books. He lives in San Francisco but was coming through London, so we ended up meeting in London rather than our California homes.

What Better World Books does is truly amazing, and deserves all the success it has had, which is considerable, and I think BookMooch is going to work with them in the future.

What they do is work with charities that run book drives, collecting all the books that remain unsold by the charity at the end of the book drive. They then find the best homes they can for those remaining books, either by selling the books (and raising money for the charity or through other charitable donations).

What this concretely means is that they list the books on eBay and Amazon and on their own website, and pay back a significant percentage of any money that comes in from those sales back to the charity. In this way, they greatly increase the amount of money a charity can receive from a book drive, and they also prevents the waste of large numbers of books. I remember reading about how a local hospital sent all the unsold books from their book drive to the trash dump, and how upset I was to learn that.

Of the books that are not sold, if it is a textbook, then it goes to Books for Africa, where extremely current textbooks are really in demand and in short supply.

They also have deals with volume used book purchasers, who will buy the remaining fiction books by the pound. That’s much better then throwing the books away, and the charity of course gets their percentage. Only in the last resort are are the books sent to be recycled.

Their numbers are extremely impressive and posted publicly on their website.

* Collected over 11.4 million books through active book drives at over 1,600 colleges and universities and collections from over 900 libraries

* Raised over $2.8 million for over 80 literacy and education non-profit organizations,

* Raised more than $1.7 million for libraries and thrift stores nationwide

* Directly sent more than 1 million books to Books for Africa, the National Center for Family Literacy, and Feed the Children

* Contributed more than $1 million to college service clubs who have run book drives

And that’s just a subset of what their web page lists.

---

BookMooch & Better World Books Partnership ideas

Xavier And I got along famously, and came up with several partnership ideas between Better World Books and BookMooch. I’d love to get some feedback on them!

Here is what we are thinking of doing:

* Giving to moochers: Better World Books would become a BookMooch member, and list books (those they cannot sell) for mooching. This helps increase the catalog of books on BookMooch. Eventually they would list almost their entire catalog of 1 million titles! Less books going to the dump, and a better selection for mooching. Initially, they’d start out small, to make sure that moochers are happy with the experience, but since their goal is to find a home for each book they receive, BookMooch is perfect for them.

* Mooching books: By earning these mooch points honestly, they also earn the right to mooch books that they can sell (or send the book to a charity they work with that really needs it). A healthy percentage of those proceeds go to charity. Since they are extremely successful at finding buyers for very obscure books, I think this will help moochers give away books that we don’t yet have an audience for, while both supporting both a business I like and raising money for charities.

* Bulk receiving of books: if you have boxes of books and do not want to bother entering them into BookMooch, Better World Books will accept whatever books you postal mail to them in whatever quantity you want, and pay you in book mooch points. What I suggested is some sort of automated pay scale like this: a) if the book title has been mooched in the past month, then pay a one mooch point, b) if the book title has been mooched in the past six months, then pay one half a much point, otherwise pay .1 mooch points. A lot of people on the forums, and in person to me, have asked if there was a way BookMooch could take a bulk shipment of books off their hands. This would of course be voluntary, and only for people who don’t want to be bothered listing books in BookMooch and sending them one at a time.

Thoughts?

---

There are a lot of comments on this blog entry, and this morning I was quite pleased to find that Xavier has replied directly on the forum. Since it’s his company being talked about, I’m also reproducing his reply here in the blog body itself, so it gets extra attention:

From: Xavier Helgesen Hi all- Thanks for all the feedback. As John mentioned, we had a great discussion, and I’m happy to be involved. We’re definitely looking forward to getting more involved with the BookMooch community and finding ways to add value. The core of our model is creating value for all stakeholders: customers, non-profits, partners, employees and anyone else we work with. I’ll try to address as any questions raised as I can - please let me know if I missed anything. Also, I encourage people to contact me personally. There seems to be a lot of conjecture about what Better World Books is and isn’t online. I’m always happy to have a chat about the company, as I feel very strongly about the positive impact we make, and I don’t see us as any threat to small booksellers. Even if we grew by a factor of 10, we’d be a very, very small part of the book market. My email is pretty easy to guess: xavier at {company URL} Cam: THANK YOU! Please keep checking with us first for affordable used books - if you email me, I’ll send you the friends & family discount code. Any others reading the comments are welcome to request this as well. Jon & Ashleigh: Thanks for your posts - that is exactly how I think is the best way for things to work. We’ll act as a responsible member of the community, just like any other. Regarding shipping books to UK & Canada: This is something that we’re looking strongly at. We’re going to roll the program out first in the USA, and proceed from there. There may be ways we can cover part/all of the cost of shipping to us, but we would have to do it based on the value of the books sent. There are a great deal of books in the world that are very hard to resell and are not of much use to our literacy partners. Advanced Readers Copies: Those are tough for us, as the publishers really push hard on reselling them, and they are not the final versions of the books. So generally we have to politely decline those books. Regarding Emily Thayer’s Points: We are a for-profit social venture, and we make no bones about it. We’re very clear about this on our website. We are not a publicly traded company however - we are independent, and owned primarily by founders and employees. We pride ourselves on providing health insurance for all employees after 90 days (which is VERY expensive for a small company like ours) and paying fair wages. We also would love to work with you on a local level - we offer warehouse pickup of books on BetterWorld.com, and we’re happy to work out bulk sales discounts and other arrangements to support our local booksellers. Just follow the bulk sales link on the site. I’m sure if you stopped by and met us, you’d see that we’re nice people, and we’re certainly not a “big business” of the ilk of Wal-Mart. We are very proud to be part of the Mishawaka community and try to engage any way we can. We encourage our employees to volunteer, and have partnerships including with the Center for the Homeless and the Robinson Community Learning Center. We have also created over 100 jobs in the community, ranging from picking orders to writing software. Regarding our participation: As I mentioned before, our participation in BookMooch only makes sense to the degree that we add value to the community. If some members want books that there is not enough demand for on our sales channels, and there are other books we can receive in return that our customers do want, that seems to be a win-win to me. As the system is very cleverly designed, we’ll only be able to receive books to the extent that we provide books that members want. I’m not sure yet whether our inventory will be a good match, but I suspect that it will. Also, the program for people to send books in will not be for everyone. We’ll have to see if it is useful, and adjust it if it is not. I can’t see how people could object to it, as it is an optional program and certainly it doesn’t hurt if it exists. Regarding Mark William’s post: Mark, you rightly mention that Books For Africa is a top-notch publicly accountable charity. The fact that our programs are both the largest source of funding and University-level books for Books For Africa ought to tell you something. We are fully transparent with our non-profit partners, and you can feel free to independently verify this and contact them. I highly encourage anyone who wants to front the postage or lives in the Minneapolis area to donate books direct to Books For Africa. I serve on the board of the organization and it has been an exceptional privilege. You do have a few facts mixed up - we sell almost all of our books on consignment for non-profits and libraries, and we pay a percentage of gross, not net. The commission varies, since we cover all costs associated with the book collections. Since we are a nationwide program, there are many costs associated with book collection and inbound shipping that are always going to be higher than a local operation. The strength of our program is really the scale - most local booksellers cannot take on 20,000 books at once and sell them effectively. They would typically have to “cherry pick” the best books and recycle the rest. Because of our scale, we are able to handle these kind of volumes, and that is a reason a lot of libraries have chosen to work with us. Finally, I can assure you that our profits are not in the 10’s of millions. As I’ve mentioned in a blog comment on another site (where they wanted to see the cars and houses of the founders), I rent and don’t own a car. We have relatively small margins after the amount we share with our literacy partners, and reinvest a great deal in our people, our software, and spreading the word about Better World Books. Sincerely, Xavier Co-Founder, Better World Books

---

John writes: I found the comment below to be really fascinating, so I’m reproducing it in the body of this blog as well:

As a BookMooch charity and a non-profit partner of Better World Books, the Prison Book Program has a unique perspective on this issue. Better World Books has been selling books we cannot use since 2004 and we have found them to be a great partner. To date they have raised over $25,000 for us which accounts for over 33% of our budget and has funded shipments to more than 8000 prisoners. We get 30% of the revenues for the books they sell on our behalf. Yes, that is relatively small, but when you consider that they pay for everything from the shipping of our books to their warehouse to the handling of the purchases, it is more than fair. We have tried selling books on our own through Amazon, through local booksellers and by running our own book sales. All proved to be far too much effort for way too little gain. BWB brings economy of scale to non-profit book selling in the same way Amazon brings it to book-selling in general. Many may not consider that a good thing, but we most certainly do. The person that manages our relationship with BWB works in the publishing industry. She has met the CEO and founders and visited their facility in South Bend, Indiana. She found them to be genuinely dedicated to funding literacy efforts and was quite impressed with their operation. They are also a recent recipient of Fast Company Magazine’s social entrepreneur award. I worked for a non-profit that received this award several years in a row. I can say that Fast Company does not award this honor lightly. The recipients are changing the world in powerful ways and are doing it honestly and efficiently. BookMooch has also been a boon to us in the ~6 months we have been an official charity. We have mooched over 350 books and nearly all of these are so in-demand that they are sent to a prisoner within a week of receiving them. Because of BookMooch we have been able to dramatically improve the quality of reading material we send out. We have mooched dictionaries (which prisoners use to teach themselves to read), almanacs, self-help and religious materials of all kinds, business books and many more. When an inmate learns to read or start a business or gets his GED or improves his mental state, he is far more likely to have a successful life on the outside – and because of legions of generous BookMoochers we are able to provide many more of these hard-to-find titles than we have in the past. Unfortunately, we (and the other prison book charities) regularly clean out BM’s inventory of these books. Having more available would be fantastic! BWB and BookMooch joining forces would make more of these titles available to everyone. We pre-screen the books we send to BWB to weed out the un-saleable ones – so I am very familiar with the books they deem un-saleable and could become available on BookMooch. While some are the same MMPs that likely are sitting in everyone’s inventories for months, a great number of them are non-fiction titles that would go in minutes on BookMooch. BWB has increased the quantity of books we send out. BookMooch has improved the quality. Working together, they may be able to do more of both. It sounds like there are many details to be thought out and considered before going forward with this plan, but I do hope that two of our favorite partners will find a way to work together. They would make a powerful team that would benefit the entire community. Marlene Cook Prison Book Program www.prisonbookprogram.org Quincy, MA

BookMooch volunteer admin Dave came up with a good idea: re-using used mailing packages, and circulating these used packages by using BookMooch itself.

Specifically, he suggested that bundles of a dozen or half-dozen mailing envelopes could be made available by people who have excess packaging, and mooched by people who need them.

Even better, if you’re mooching a book from Dave, or any other user with used packaging, you should also mooch some used mailing packages, so they can include them in his mailing to you, and there’s no extra mailing cost for the sender.

You can find the mooch packages by searching for the word “moochpackages” like so:

if you have your own packages to give away, click on the “dozen” or “half-dozen” link, and then click the “give your copy” link:

A brand new Facebook widget for BookMooch has been written, and it has some really nice features.

To add the new BookMooch widget to your page, go to this url:

http://facebook.com/apps/application.php?id=9914231158

You should see a screen like this below, from which you can click the “add application” button on the top right of the page. You might need to log into Facebook first before you see this page.

you’ll then have to configure the widget to do what you want. My preference is to show 7 recent book covers I’ve wishlisted, mooched or am giving away, because 7 covers is what fits on one line in Facebook. I’ve some problems with the “Random covers” choice today (but it works for Kevan in the UK), so there if that doesn’t work for you, try “recent covers” instead. Be sure to give your BookMooch user name (the one you use to log in with). No password is necessary, since this is publicly viewable information on the BookMooch site anyway.

Your Facebook page will now list information about your BookMooch activities:

Even cooler, books you mooch or give away show up on your minifeed, so that your friends can see what books you’re offering, and what you’re planning on reading:

Kevan Davis, a moocher in the UK, wrote this app. It’s still in beta, and might have some problems still. We’re looking for feedback and ideas. One idea I had, that Kevan thinks is silly (grin) is to show books that I recently gave away. Kevan thinks that’s a “tease” (see what you missed out on, ha ha) and I think it’s interesting to see what my friends are getting rid of. Kevan’s probably right (feature overkill) but I’d love feedback for other features, or ways this could work differently.

---

Tell your Facebook friends about the BookMooch widget

The easiest way is to share the widget is to click on the “bookmooch” link on your own profile page:

and then on the top right of the settings page, click “share”

Or you can go to this url:
http://www.facebook.com/apps/application.php?id=9914231158#

click on the “share” button on the right column:

A popup appears that lets you post a message to your wall:

here it is on my wall. Notice how the wall has a “share” button, which makes it *much* easier.

note that you can also send email messages to your friends about the Facebook widget.

---

There is a BookMooch group of Facebook, which you can join at this URL:
http://www.facebook.com/group.php?gid=2209177001

In response to my blog entry about “publishers giving BM some green love” a discussion sprang up in the comments of that blog entry, about how “green” book-trading-by-mail is.

I’m definitely sympathetic to the concerns. However, books are a physical product, and as such have an energy, resource, waste and pollution footprint.

If we like books (and I do) and we want to be “green” I think our goals should be:

1. to have fewer copies of books printed
2. to have those copies be re-read as much as possible
3. and to be as efficient as possible in moving the books from reader to reader.

As such, some long term goals I have for BookMooch, which I want to mention now, are:

1. make book trading so effective that publishers can print fewer copies of a book, and yet reach a larger audience. (to accomplish this, I have to find a way for publishers to profit from BM)
2. promote “meet & mooch” get-togethers, where people regularly meet and bring books the others attending indicated an interest in, so that books can be felt, browsed, discussed and mooched in person.
3. provide a “mooch shelf” where books being traded can be dropped off and picked up at your leisure (and safely) in a place you’re already driving by. These could be at libraries, schools and cafes.
4. encourage multi-mooching, i.e. getting several books from one person. This lowers postal costs and the carbon footprint per book.
5. encourage use of bookmooch among real-world friends who live near each other. This will be through a “friends only inventory” feature I’m planning, where you can list books that only your friends can see and mooch, as well as a friends-only discussion group.

I’m sure more good ideas will emerge over time. These are all things I want to focus on over the next year, as they help improve BookMooch overall by:

1. lowering the postage cost for bookmooch members
2. lowering the cost of being a book publisher
3. foster communication, community and friendship among book readers
4. integrate book reading into existing real-world structures (cafes, libraries, schools, etc…)
5. increase the rate at which unwanted books find a new, happy home

A few days ago, my friend Cory Ondrejka suggested something along these lines, namely promoting local mooching more, and the use of Starbucks as a meeting place where two local moochers could exchange the book face to face. There are a lot of pros to that idea (no postage cost, drinking more coffee, meeting people) but it’s also complicated (which Starbucks? There are so many! And when should we meet?). Of course, this wouldn’t be just Starbucks, but any open meeting place (such as an indie bookstore or cafe). Another angle is to use the “local book events” feature at LibraryThing as a meeting-place. I’m not sure this idea will work yet at BookMooch, because moochers don’t yet frequently live near each other, but it might work for certainly major cities.