Two of my favorite webcomics doesn’t have official feeds - Malvados and The Order of The Stick (this one actually *have* a feed, but the images aren’t shown in the feed items). I didn’t like to visit these sites manually, so I quickly wrote a script (which is now a very simple Ruby lib) to create these feeds for me: rssifier.

The lib code is ugly at some points, but it’s working for me. Here is an example script (malvados.rb) that uses rssifier to create a feed:

(Link of the Gist).

And this is the result; I have a cronjob calling the malvados.rb periodically, so this XML gets updated from time to time.

Any contributions via github are obviously welcome :-).

I’ve been working on a Django project lately and I had the need to send some emails from my application. I was trying to figure out how to write a mocked version of django.core.mail.send_mail when I read about django.core.mail.outbox, which apparently is a new feature of Django 1.0.

django.core.mail.outbox is a list of all instances of emails sent with django.core.mail.send_mail that is available only in the testing framework. With it you can do things like these:

(Link of the Gist)

Dois lançamentos recentes estão deixando o povo que ‘mexe com web’ agitado: o Mozilla Ubiquity e o Google Chrome.

Ubiquity

O Ubiquity é um ‘Quicksilver-do-Firefox’, uma ferramenta que permite que usuários manipulem diversas ferramentas web usando uma interface de texto simples. Por exemplo, clicando Ctrl+Space (para abrir a interface de comando) e digitando ‘map criciuma,brazil’ verá um google map mostrando a cidade onde eu moro. Essa é uma *excelente* ferramenta para agilizar o uso de diversas ferramentas da web - além do google maps do exemplo acima, na versão 0.1 do Ubiquity já existem comandos para fazer buscas na Amazon, no Flickr, definir termos, fazer cálculos e fazer muitas outras coisas.

Uma das coisas que eu mais achei interessante é a facilidade de manipular conteúdo; você pode, por exemplo, selecionar um texto qualquer em uma página, abrir o ubiquity e digitar ‘email this to fulano’ - isso irá abrir uma tela do gmail já com um email parcialmente preenchido para o ‘fulano’ (que deve estar nos seus contatos).

Chrome

O Chrome é o novo browser do google. É um browser que usa o engine Webkit para renderizar as páginas e um novo engine javascript (o v8), bem mais rápido que o dos browsers atuais. O Chrome foi pensado na web atual, não na web de tempos atrás: a web atual é uma plataforma de aplicações. GMail, GDocs, Basecamp, Twitter, Github, GReader, GCalendar, Facebook… parando pra pensar, eu praticamente não entro mais em ’sites’ - mesmo os blogs que leio são sempre via RSS.

O Chrome foi desenhado pensando nessa ‘nova web’: entre outras coisas, cada aba é um processo. Isso faz com que esse navegador seja mais seguro (uma aba não consegue acessar os dados das outras), mais rápido (uma aba lenta não afeta as outras) e trave menos (se uma aba travar, é só ela que vai travar). A interface é minimalista: a idéia é que o navegador seja ‘invisível’, que o usuário só preste atenção no conteúdo que está acessando.

“Mas como assim p*rra? O Google e a Mozilla não tinham um ‘caso’?”

Pois é. Como bem disse o Sidnei, o Chrome tem tudo pra ‘roubar’ usuários do Firefox. De qualquer forma, a bagaça é open source - se a Mozilla quiser pode ‘deitar e rolar’ com o código do Chrome - inclusive pode aproveitar e começar a usar a engine javascript do novo browser, que é bem mais rápida que a do Firefox.

Qro nstala n Linx comofas/

Perdeu, preibói. Por enquanto o Ubiquity só roda em Windows e Mac OS X, enquanto o Chrome só em Windows.

E vejam que beleza: migrei do OS X de volta pra Linux há menos de uma semana. O jeito é usar o VirtualBox.

*sigh*

Nos últimos dias venho trabalhando em uma app Django e, como sou iniciante no uso desse framework, todos os dias aprendo algum feature novo para melhorar meu código (como generic views, por exemplo). Para testar esses features o ideal é criar um branch, usar os features no código e, se valer a pena, fazer o merge. O problema: usamos svn na empresa, e fazer branches no svn é bem chatinho (principalmente se comparado ao git). Para resolver isso estou usando o git-svn.

Segue abaixo meu fluxo de trabalho no git-svn:

Link do Gist, para quem não consegue ver no feed.

Enfim, esse é um post pra que eu mesmo não esqueça como fazer isso. Alguns links e tutoriais interessantes sobre o git e o git-svn estão no meu del.icio.us: http://delicious.com/dirceu/git.

As the Google Summer of Code coding period comes to an end, I must write about what I have done, so here we go. My proposed tasks were:

1. Invalidation processing optimization: test if ZEORaid is processing validations from the backend storages too often. This should be proven and maybe optimized.
2. Allow defining/removing new backend storages while running using the client tool or reloading the config.
3. Define a transaction rate limit / recovery ration to ensure that recovery can always be faster than new transactions that are flowing in.
4. Currently requests to multiple backends are run in sequence. This should be done in parallel, to speed up.
5. Single requests to backends should be distributed (randomly or round robin) over the available backends.
6. Implement `off-site` backends which are written to as a kind of secondary backend which don’t have to respond to write requests synchronously to allow replication over slow links to a different site.

Task 1

The first two weeks were used on this task and on getting myself familiar with gocept.zeoraid codebase. As I said in my first report, I didn’t found anything wrong with the invalidation proccess when doing my tests, so I moved to another task.

Task 5

This was a reasonably easy one. My main problem was getting an automated test to prove the implementation. It’s hard to test a distributed system, and doing the setup is a little problematic; anyway, with help from my mentor, Christian Theune, I got a working test. My work is on this branch.

Task 4

This one was tricky. I had no “real world” experience on writing threads, so I stepped into many obvious bugs - joining threads, dealing with exception handling and such things. I’ve spent almost four weeks working on this task, but in the end I got it right. My work is on this branch.

Task 3

I started researching a bit about this task, but Christian said that task 2 was much more important, so I stopped working on it.

Task 2

My final task over the GSoC period. I started by writing a “add” command to the ZEORaid command-line tool, as I described in this report. It was easy to get this working for ClientStorage objects, but I needed to get it working with other types of storages too; some time later Christian said that it was only a “helper” for me to understand better the way that ZEORaid handles storages and openers.

I dropped this implementation and started working on reloading the entire config file. I got it working in a couple of days, but spent *much* more time than that working on an automated test for it. My work is on this branch.

Quick report: this week I’ve finished the ‘reloading’ task - now with automated tests, thanks to the help of Christian, my mentor. Now I need to document my branches and write a post about what I’ve done on the coding period. I need to do this on this weekend, because next monday is ‘pencils down’ date, when all mentors must submit a final evaluation of the work done by their students.

I’ve nearly finished the ‘reloading’ task described in my last report; now we’re able to reload the config file without modifications. The code is quite simple: we open and parse the config file, using the list of storages defined there to see what storages needs to be removed (and then we disable them) and what storages needs to be added (and then we add and ‘recover’ them). I’m now working on a automated test to prove the implementation is right.

Published on August 9, 2008.

Just a quick report: this week I worked on reloading the zeo.conf file, so we can add or remove storages from the ZEORaid server without having to restart it. It seems to be working well, but it needs better (automated) testing and I need to use the correct ZODB.config options to recognize the config file without problems (this has been my main issue for the week). My work is on this branch.

Published on August 2, 2008.

This week I’ve been working on allowing to define and remove new backend storages while running using the management tool or reloading the config. I worked first on defining a new storage using the management tool. Using this branch, you can connect to a new ClientStorage using ‘zeoraidX-manage-main add IP:PORT:STORAGE_NAME’. Example:

# ./bin/zeoraid1-manage-main add 127.0.0.1:8003:main

Now I need to work on the following things:

• Add any kind of storage, not only ClientStorage;
• Add a command to remove an existing storage using the client tool;
• Add a command (’reload’) to reload the zeo.conf and add/remove the storages as needed.

Published on July 26, 2008.

I’ve been working on the task I described on my last post, about parallelizing request to multiple backends. After many time getting strange errors on the tests I got to isolate the problem - my code wasn’t handling the exceptions right.

ZEORaid considers ZODB.POSException.POSError and transaction.interfaces.TransactionError valid answers from storages, as they don’t indicate storage failure. The problem was that I was getting some ConflictErrors (a kind of TransactionError) that were being considered a sign of storage failure when running ZODB tests.

Christian helped me to work on hadling this and now all tests are passing. I’m just refactoring the code now and will procceed to the next task: allow defining/removing new backend storages while running using the client tool or reloading the config.

Published on July 18, 2008.