Why muck around with HAProxy and Varnish when you can have nginx do it all for you. The setup is easy and it's a lot easier to maintain.

Installing nginx with buildout

You can setup nginx fairly easily with buildout. The only fancy part of our setup is that we're going to include the nginx cache purge module.

1. Add the cache purge part to your buildout
   

   [ngx_cache_purge]
   recipe = hexagonit.recipe.download
   url = http://labs.frickle.com/files/ngx_cache_purge-1.1.tar.gz
   strip-top-level-dir = true

2. I needed the pcre source to compile also
   

   [pcre-source]
   recipe = hexagonit.recipe.download
   url = ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.00.tar.gz
   strip-top-level-dir = true
   

3. and the nginx part
   

   [nginx-build]
   recipe = hexagonit.recipe.cmmi
   url = http://nginx.org/download/nginx-0.8.45.tar.gz
   configure-options =
       --with-http_stub_status_module
       --conf-path=${buildout:directory}/settings/nginx.conf
       --error-log-path=${buildout:directory}/var/log/nginx-error.log
       --pid-path=${buildout:directory}/var/nginx.pid
       --lock-path=${buildout:directory}/var/nginx.lock
       --with-pcre=${pcre-source:location}
       --with-http_ssl_module
       --add-module=${ngx_cache_purge:location}
   

4. and finally, add it all to the parts directive
   

   parts =
       ...
       pcre-source
       ngx_cache_purge
       nginx-build
       ...
   
   

5. After you re-run buildout, you'll be able to run nginx by issuing a command like this:
   

   ./parts/nginx/sbin/nginx -c /path/to/configuration/nginx.conf

nginx configuration

Here is a simple sample configuration for nginx configured with load balancing and caching. It can obvious get as complicated as you want it, but I definitely think this is easier than managing haproxy, varnish and nginx.

pid /path/to/buildout/var/nginx.pid;
lock_file /path/to/buildout/var/nginx.lock;

worker_processes 2;
daemon off;

events {
    worker_connections 1024;
}

error_log /path/to/buildout/var/log/nginx-error.log warn;

# HTTP server

http {
    server_names_hash_bucket_size 64;

    # this is how you do simple round robin load balancing with nginx.
	# you can define as many backup servers as you'd like here.
    upstream plone {
        server 127.0.0.1:8080;
        server 127.0.0.1:8081;
    }
    
    access_log /path/to/buildout/var/log/main-access.log;


 	# can specify multiple cache paths for different resources/paths/proxies
    # if needed..
    # the levels=1:2 just means it'll store the cache'd files 2 levels down in
    # the folder structure
    proxy_cache_path  /var/www/cache  levels=1:2 keys_zone=thecache:100m max_size=1000m inactive=600m;
    proxy_temp_path /var/www/cache/tmp;

    # Here is the caching purge handling. Purge request come in here
    server {
        listen 8089;
        server_name www.example.com;

        access_log /path/to/buildout/var/log/purge.log;

        location / {
      	  allow			127.0.0.1;
      	  deny			all;
      	  proxy_cache_purge thecache $scheme$proxy_host$request_uri;
        }
    }
    
    server {
        listen 80;
        server_name www.example.com;


		# log for cache hits.
		log_format cache '***$time_local '
		                             '$upstream_cache_status '
		                             'Cache-Control: $upstream_http_cache_control '
		                             'Expires: $upstream_http_expires '
		                             '"$request" ($status) '
		                             '"$http_user_agent" ';

		access_log /path/to/buildout/var/log/cache.log cache;

        # Enable gzip compression
        gzip             on;
        gzip_min_length  1000;
        gzip_proxied     any;
        gzip_types       text/xml text/plain application/xml;

        # Show status information on /_main-status
        location = /_main_status_ {
            stub_status on;
            allow 127.0.0.1;
            deny all;
        }

		# do not cache when users are logged in..
        proxy_cache_bypass $cookie___ac;

	    location / {
	        proxy_redirect                  off;
			proxy_set_header                Host $host;
			proxy_set_header                X-Real-IP $remote_addr;
			proxy_set_header                X-Forwarded-For $proxy_add_x_forwarded_for;
			client_max_body_size            0;
			client_body_buffer_size         128k;
			proxy_send_timeout              120;
			proxy_buffer_size               4k;
			proxy_buffers                   4 32k;
			proxy_busy_buffers_size         64k;
			proxy_temp_file_write_size      64k;
			proxy_connect_timeout           75;
			proxy_read_timeout              205;   
	        http://plone/VirtualHostBase/http/www.example.com:80/Plone	/VirtualHostRoot/;

	        proxy_cache_bypass $cookie___ac;
	        proxy_cache thecache;
	        proxy_cache_key $scheme$proxy_host$request_uri;
	    }
    }
}

Sometimes grep or TextMate's find can take a long time so I decided to give a mini search engine a try using repoze.bfg, repoze.catalog and Pygments. The result is that it works rather well and is very fast. The catalog ends up being rather large once you add groups of packages from Plone and such so this ends up making the RAM usage be very high unfortunately.

It gives you the ability to easily search methods, classes, filenames and full text and then give allows you to view the file with pygments.

You can take a look at search.nathanvangheem.com to see it working with repoze.bfg and Plone 4 being indexed.

You can checkout the pypi page if you're interested in it and I also have it on bitbucket.

Tres has managed to merge his WSGI branch into trunk and Hanno tells me the unofficial plan is to include this in a release for Zope 2.13, in time for Plone 4.1. This is not decided upon yet though.

Overview

I was planning on implementing WSGI for Zope2 during the Penn State Symposium Sprints, and I did a few things to help out; however, Tres Seaver did most of the work on his own before and during part of the Sprint :) The rest of the time I spent just testing it out and helping with the Theme Editor sprint.

Now, it's really quite trivial to get it working now and makes all of the repoze.zope2 nonsense unneeded now. This article is just here for a reference if anyone else is interested in getting it going on their setup.

Guide

This guide assumes you have an existing Plone 4 installation to work from. I don't provide any buildouts here--just modifying an existing buildout to make it work with a branch of Zope2 and creating an ini that Paste can consume to serve WSGI.

Supplying the WSGI'd Zope2

First off, go to the src directory of the installation. If you've installed using the unified installer, that will bin in instance-home/zinstance/src or if you just used straight buildout, it'll be in instance/src. Then checkout the Zope2 branch:

svn co http://svn.zope.org/repos/main/Zope/branches/tseaver-fix_wsgi/ Zope2

Stringing up buildout

Next thing you'll need to do is modify your buildout.cfg file to add the checked out Zope 2 to the develop section:

develop = 
  ...
  Zope2
  ...

Still modifying your buildout.cfg, add Paste, PasteScript, repoze.tm2 and repoze.retry to your eggs section:

eggs =
  Plone
  Paste
  PasteScript
  repoze.tm2
  repoze.retry 

Again, editing your buildout.cfg, add a paster part for the paster script:

parts =
  ...
  paster
  ...

[paster]
recipe = repoze.recipe.egg
scripts = paster
eggs = ${instance:eggs}

Then, you'll need to add the updated Zope2 versions for the WSGI branch. To do this, basically, just add the versions.cfg file provided in the branch after every other version file listed in the extends directive. It'll look like this:

extends =
  ...
  src/Zope2/versions.cfg

Then run your buildout like normal:

./bin/buildout

Creating a WSGI Configuration File

You'll now need to create a WSGI configuration file. Right now, we'll just server it using the Paste server and wsgi ini configuration way. You can also do this to string up Apache's WSGI implementation but that is beyond the scope of this article.

Create a file in the instance directory called, zope2.ini with the contents of:

[app:zope]
use = egg:Zope2#main
zope_conf = %(here)s/parts/instance/etc/zope.conf

[pipeline:main]
pipeline =
    egg:paste#evalerror
    egg:repoze.retry#retry
    egg:repoze.tm2#tm
    zope

[server:main]
use = egg:paste#http
host = localhost
port = 8080

The zope_conf value in the app:zope section can be the path to any zope.conf file. I'm just exampling the standard location of it and not going through the configuration of that file itself.

Fire it up!

If all went well, you should now be able to start up your Plone 4 instance on WSGI like this:

./bin/paster serve zope2.ini

Your server should now be able to visit your site on http://localhost:8080

Caveats

I did run into a snag with the Mac OS X unified installer and the version of python it has configured. Basically, it wouldn't compile the Zope2 dependencies so I had to use my own version of python that I had compiled with the python buildout found in the plone collective svn. The bug is sort of referenced in the zope bug tracker.

Future Considerations

I'm hoping to maybe get a release with this branch implementation out--maybe as an alpha or beta release since I don't think they are planning on merging this to core any time soon; although, I really have no understanding of what that whole process is.

I'd like to see the Zope2 package implement mkzope2instance and other convenience methods so it'd be possible to install Plone 4/Zope2 without buildout at all maybe using pip. I'm looking into how this might be able to happen with a pip versions file and other things. Maybe more on this later.

Post any comments if you run into any issues.

Snags

• the search didn't work because of this migration issue http://dev.plone.org/plone/ticket/10360
• old file types aren't migrated to blob http://dev.plone.org/plone/ticket/10365
• archetypes don't want to use the new add views--should probably report this
• If you have the jquery UI package installed, it'll mess with some of Plone's JS for some reason--I just disabled it.
• some other control panel entries and menu items were not fully migrated, still pointing to old templates

Benefits

• FAST--running this on a linode with 512MB of ram. 2 zeo clients and with CacheSetup, this thing is very fast. Much faster than Plone 3.
• Just more polished than Plone 3--with usability enhancements and a new theme, it's great.
• I haven't found a product that wasn't compatible with Plone 4 yet--although, sometimes I needed to do some digging or use an svn checkout to get the compatible version

The fix is to remove all sub-requests in the deliverance rules file if possible, add more threads threads in the repoze.zope2 wsgi server configuration, or use a different wsgi server that doesn't restrict the amount of threads--so it'll add a new thread if needed. A better solution, IMO, would be to just have a different server, serve the theme files. Anything to make it not make sub-requests against the same wsgi server.

First off, find a nice fancy eject button and place it into boxee's skin images folder, "skin/boxee/media/icons/". I used this one and found it on google image search.

Next insert these lines after line number 231 of the file skin/boxee/720p/boxee_main_menu.xml

<item>
    <visible>System.HasMediadvd</visible>
    <controlid>1300</controlid>
    <thumb>icons/eject.png</thumb>
    <label>Eject DVD</label>
    <onclick>XBMC.EjectTray()</onclick>
</item>

And you're set. You should now have an eject button for your DVDs.

You can download my buildout here. It also contains patches for it to work on Mac OS X. You'll need python installed for this to work.

If you're not familiar with buildout, these are the steps to get it working after you've extracted it...

python bootstrap.py
./bin/buildout

Then wait for a long time. Then read the README.txt for instructions on how to configure and start apache

collect2: ld returned 1 exit status
lipo: can't open input file: /var/folders/nO/nOf0lGB+GlGrFGnhcyaFPE+++TI/-Tmp-//ccqHwR4o.out (No such file or directory)
error: Setup script exited with error: command 'gcc' failed with exit status 1

Then it's failing because it doesn't have the universal versions of some packages installed. 

You'll first need to install MacPorts. You can download the dmg install file or just go to the MacPorts installation page for instructions on how to get it setup on your system.

Installing Universal Packages

sudo port install libiconv +universal
sudo port upgrade --enforce-variants zlib +universal
sudo port install jpeg +universal
sudo port install freetype2	 +universal

Now Install PIL 

./bin/easy_install http://dist.repoze.org/PIL-1.1.6.tar.gz

 This is a version of PIL that plays nice with setuptools.

• Picasa albums have 1000 photos limits
• the python gdata api didn't support video uploads
• Picasa trimmed whitespace from Album names
• other small things

Moving

easy_install flickrapi
easy_install gdata

On this server, I run my web site and a few free sites that I do for friends and family. Previously, I was just running a zope instance for Plone and a couple separate Paste served apps. It is also running Ubuntu and nginx on a Linode shared host.

updates

While working with a client on Deliverance and repoze.zope2, I realized it was about time to eat my own dog food and implement the new technologies on my own server. Since I've done a lot of the implementation and research for my client already, it was pretty easy to move over to the new setup. The setup was closely borrowed from Martin Aspeli's uber bulidout in the plone collective svn. Here are some quick notes on updates of the setup:

• repoze.zope2
• Deliverance
• wsgi
• nginx
• new site design themed with Deliverance
  
• everything built using buildout include lxml and nginx
• move to zeo cluster with only one back end. This allows me to attach debug instances to the zeo server and turn one client on while the other is restarting.
• Zine blogging software built in
• supervisor managed - supervisor allows you to monitor processes and http servers. It includes starting and stopping processes, tools to monitor the memory and availability and a web server to monitor them.
  

highlights

• collective.recipe.template is a great buildout recipe for various tasks to generate configuration files for nginx served sites configuration and Paste serve ini files
• collective.recipe.backup provides easy to use backup and restore scripts for ZODB storage
• collective.recipe.filestorage easily adds new mount point configuration to the plone.recipe.zope2zeoserver and plone.recipe.zope2instance recipes
• collective.recipe.supervisor add supervisor to the setup in a configurable way
• z3c.recipe.staticlxml allows you to compile lxml statically
• hexagonit.recipe.cmmi provides mechanisms to easily build non-python software
• repoze.zope2 allows me to serve Plone via a wsgi pipeline, which also allows me to add Deliverance to that pipeline
  
• zinebuildout integration so I can, with ease, provide multiple zine blogs
  

 conclusion

Overall, it was a great update. The only thing I'm concerned about is the size of my virtual host, which only has 512 MB of RAM right now. So far it seems to be holding up okay, but I'm not running Varnish or load balancing(I don't get that many hits)....

Features

• No more Gallery type = getting prepared for dexterity
• custom gallery and display type story is much improved so it is very simple for people to add on to the product
• the classic display type is removed
• the addition of fancybox and highslide display types
• more visually pleasing through out
• un-intrusive
• Collections are now supported!

 

First thing, in any folder or collection, change to the "Gallery View" 

Once the "Gallery View" is selected, a "Gallery Settings" tab is enabled. You can clicked there to start customizing your gallery.

The default display type for galleries is Slideshow 2 now. 

This release also contains Fancybox display type with batching.

and finally, Highslide

 

For more information, check out the pypi entry.

• An event is just an interface definition and the event object declaration
  

  import zope.component.interfaces
  from zope.interface import implements
  
  class IMyNewEvent(zope.component.interfaces.IObjectEvent):
      """An object has been modified"""
  
  class MyNewEvent(zope.component.interfaces.ObjectEvent):
      """An object has been modified"""
      implements(IMyNewEvent)
  

• The real magic happens when fire off the event
  

  from zope.event import notify
  
  notify(MyNewEvent(context))

• Now to subscribe to the event do something like this,
  

  <subscriber
    for="myproduct.myobject
         myproduct.interfaces.IMyNewEvent"
    handler=".subscribers.event_handler"
    />

• Now in a file subscribers.py add the method event_handler like this,
  

  def event_handler(obj, event):
    # do something interesting here...
    pass

    
  

• tool.py
• interfaces.py

from zope.interface import implements
from zope.component import getMultiAdapter
from plone.memoize.view import memoize, memoize_contextless

from Acquisition import aq_inner
from Products.CMFCore.utils import getToolByName
from Products.Five.browser import BrowserView

from interfaces import ITool

class Tool(BrowserView): 

  implements(IUWOshTools)

  @memoize_contextless
  def hello_world(self):
    return "Hello, World!"

from zope.interface import Interface

class ITool(Interface):
  def hello_world(self):
    pass

<browser:page
  name="tool"
  for="*"
  permission="zope2.View"
  class=".tool.Tool"
  allowed_interface=".interfaces.ITool"
  />

<!-- We need to make the request annotatable for this to work -->
  <class class="ZPublisher.BaseRequest.BaseRequest">
    <implements interface="zope.annotation.interfaces.IAttributeAnnotatable" />
  </class>

here/@@tool/hello_world
python: here.restrictedTraverse("@@tool").hello_world()

from zope.component import getMultiAdapter

tool = getMultiAdapter((self.context, self.request), name=u'tool')