Site News

Post

Document Viewer Integration in Plone

Presenting the collective.documentviewer package for plone that allows you to display PDFs and other office documents.

collective.documentviewer integrates the great New York Times Document Viewer into Plone.

Features

  • OCR
  • Searchable on OCR text
  • works with many different document types
  • plone.app.async integration with task monitor
  • configuration options
  • PDF Group view for display groups of PDFs

Installation

There is an extensive set of system installation requirements that you must install in order for document viewer to work correctly. Additionally, it is recommended that you install and setup plone.app.async along with this package.

How it works

The docsplit tool is used to generate images and text files for each PDF. The viewer is simply just a viewer of images and text files so it's easy to style and customize. The downside of this is that, for every PDF page, 4 files are generated. For sites with a lot of large PDFs, even with blog storage, it's a lot of extra data the zodb has to manage. That is why basic file storage is also available.

The OCR text is also indexed locally with the PDF(using repoze.catalog) and globally with the plone catalog. This is done because a custom index is required for document viewer in order to search text in the PDF.

Configuration Options

After product activation, there will be a control panel item, "Document Viewer Settings."

  • Image sizes -- Customize the size of images generated for the viewer
  • Storage type -- Allows you to setup file storage for your PDF data
  • OCR -- by default, this is off because it can take quite some time to OCR documents(and with no plone.app.async installed, some users could end up being very unhappy)
  • Detect Text -- detect if text is already found on PDF, if so, do not OCR
  • Auto select layout -- automatically, for PDF and any enabled document types, select the document viewer layout
  • Auto layout type -- the types of files that should also be enabled for the document viewer layout.

A Tour

With screenshots, I will go over the various features.

Settings

Make sure to customize any settings before you start using it. Also, make sure to activate any office formats you'd like to be able to use:

ScreenShot20120429at5.29.39AM.png

The Viewer

Document Viewer Viewer

Document Viewer Pages

Document Viewer Search

Group View

A view is also added for folders and collections to display groups of PDFs and search within the groups.

Document Viewer Group View

Converting Office Documents

Office documents are also then able to be converted for the viewer.

Document Viewer Excel Doc

Async Integration

How plone.app.async integration is managed.

You can view the current status of your conversion async task by clicking the "Document Viewer Convert" button at any time. Or, if it isn't converting, you can reinitiate conversion there.

Where you can initiate conversions and see status.

You can also monitor all tasks currently in the queue:

Document Viewer Async tasks

What's left

  • It's not internationalized at all. Apologies to non-english plone users.
  • Better mobile viewer
  • If you're converting a lot of documents at once with plone.app.async, there seems to be a issue with conflict errors. Unfortunately, this could cause your document to be converted more than once..
Post

collective.plonetruegallery 2.x demonstration

Demonstrating some of the features of the collective.plonetruegallery 2.x series.

It's been a long time since I've made a blog post and this still I thought I'd try a webcast. Well, not really a webcast but a video demonstration of some of collective.plonetruegallery's features. collective.plonetruegallery was one of my first projects for plone and has matured over the years now with lots of help from the community.

New Features

  • New gallery display type integrations(supports 8 total now)
    • galleria
    • nivo slider
    • nivo gallery
    • pikachoose
    • s3slider
  • Better inline gallery support
  • Gallery portlet can now show full gallery(useful when using in conjunction with Content Well Portlets)
  • Products.Collage support

Documentation for installation and installing different display types can be found on pypi and plone.org.

The music choice doesn't quite fit for the video. Sorry folks, I didn't know what else would be appropriate. It's better than silence :)

Post

High Availability Varnish Configuration for Plone

How to get varnish to continue serving out stale, content when your backend may be down.

Why

There are many reasons why a backend server could go down or be unresponsivw and there is no reason that your caching proxy can't serve out stale content while it is down or slow to respond.

How

There are a few tricks that will help you get better performance out of varnish and that will trick varnish into serving stale content instead of an error.

Serving Stale Content

Restart the request and have varnish use an always down server on error so that it'll serve stale content right away

  1. Setup the fake backend
    ...
    backend failapp { 
      .host = "127.0.0.1"; 
      .port = "9999"; 
      .probe = { 
        .url = "/hello/"; 
        .interval = 12h; 
        .timeout = 1s; 
        .window = 1; 
        .threshold = 1; 
      } 
    }
    ...
  2. Set the grace period on the request in vcl_recv
    ...
      if (!req.backend.healthy) {
        set req.grace = 1d;
      } else {
         set req.grace = 15m;
      }
    ...
  3. Set grace period for response in vcl_fetch
    ...
    set beresp.grace = 10d;
    ...
  4. Set a marker error header in the vcl_error section and restart the request
    ...
    sub vcl_error {
      /* set a marker on so we know there is an error with the backends
         and that we should serve out stale content */
      if ( req.http.X-Varnish-Error != "1" && req.request != "PURGE" && req.restarts == 0) {
        set req.http.X-Varnish-Error = "1";
        return (restart);
      }
    }
    ...
  5. Check for the marker error header in the vcl_recv and set to already down backend
    ...
      if (req.http.X-Varnish-Error == "1") {
        set req.backend = failapp;
        unset req.http.X-Varnish-Error;
      } else {
        set req.backend = plone;
      }
    ...
    

Cleaning Up The URL

There is no need to cache the different hash urls(#) or different query parameters for google analytics

...
  if (req.url ~ "\#") {
    set req.url=regsub(req.url,"\#.*$","");
  }
  # Strip out Google related parameters
  if(req.url ~ "(\?|&)(utm_source|utm_medium|utm_campaign|gclid|cx|ie|cof|siteurl)=") {
    set req.url=regsuball(req.url,"&(utm_source|utm_medium|utm_campaign|gclid|cx|ie|cof|siteurl)=([A-z0-9_\-\.%25]+)","");
    set req.url=regsuball(req.url,"\?(utm_source|utm_medium|utm_campaign|gclid|cx|ie|cof|siteurl)=([A-z0-9_\-\.%25]+)","?");
    set req.url=regsub(req.url,"\?&","?");
    set req.url=regsub(req.url,"\?$","");
  }
...

Full Example Configuration

In this configuration, keep some things in mind:

  • The configuration is manually setting the cache age on these objects and relying more on purges to handle cache refreshes
  • The configuration assumes the public site is not for logging in, so no cookie handling is happening
  • The configuration sets additional response headers so you can see information on how varnish handled the response(ttl, grace, status, hit)
  • This exact configuration is cleaned up from what I actually use in production and you'll need to clean it up and implement your own parts of it to an extent. Please don't assume that this is just a drop in replacement.
acl purge {
  "localhost";
  "127.0.0.1"; /* and everyone on the local network */
  "10.10.10.10";
}

/* failapp is used to help trick varnish into using stale content */
backend failapp { 
  .host = "127.0.0.1"; 
  .port = "9999"; 
  .probe = { 
    .url = "/hello/"; 
    .interval = 12h; 
    .timeout = 1s; 
    .window = 1; 
    .threshold = 1; 
  } 
}

backend cms1 { 
  .host = "10.10.10.1"; 
  .port = "8080"; 
  .connect_timeout = 10s; 
  .max_connections = 30; 
  .first_byte_timeout = 300s; 
  .probe = { 
    .url = "/"; 
    .interval = 3s; 
    .timeout = 3s; 
    .window = 5; 
    .threshold = 2; 
    .initial = 1;
  } 
}
backend cms2 { 
  .host = "10.10.10.1"; 
  .port = "8081"; 
  .connect_timeout = 10s; 
  .max_connections = 30; 
  .first_byte_timeout = 300s; 
  .probe = { 
    .url = "/"; 
    .interval = 3s; 
    .timeout = 3s; 
    .window = 5; 
    .threshold = 2; 
    .initial = 1;
  } 
}
backend cms3 { 
  .host = "10.10.10.1"; 
  .port = "8082"; 
  .connect_timeout = 10s; 
  .max_connections = 30; 
  .first_byte_timeout = 300s; 
  .probe = { 
    .url = "/"; 
    .interval = 3s; 
    .timeout = 3s; 
    .window = 5; 
    .threshold = 2; 
    .initial = 1;
  } 
}
backend cms4 { 
  .host = "10.10.10.1"; 
  .port = "8083"; 
  .connect_timeout = 10s; 
  .max_connections = 30; 
  .first_byte_timeout = 300s; 
  .probe = { 
    .url = "/"; 
    .interval = 3s; 
    .timeout = 3s; 
    .window = 5; 
    .threshold = 2; 
    .initial = 1;
  } 
}

director plone round-robin {
  { .backend = cms1; }
  { .backend = cms2; } 
  { .backend = cms3; } 
  { .backend = cms4; } 
}

sub vcl_recv {
  if (req.http.X-Varnish-Error == "1") {
    set req.backend = failapp;
    unset req.http.X-Varnish-Error;
  } else {
    set req.backend = plone;
  }
  if (req.request != "GET" &&
      req.request != "HEAD" &&
      req.request != "PUT" &&
      req.request != "POST" &&
      req.request != "TRACE" &&
      req.request != "OPTIONS" &&
      req.request != "DELETE" &&
      req.request != "PURGE") {
    /* Non-RFC2616 or CONNECT which is weird. */
    return (pipe);
   }

  if (req.request != "GET" && req.request != "HEAD" && req.request != "PURGE") {
    /* We only deal with GET and HEAD by default */
    return (pass);
  }

/* Time to mess with the request */
  unset req.http.Cookie;
  unset req.http.User-Agent;
  unset req.http.Accept-Charset;

  if (req.http.Accept-Encoding) {
    if (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg|pdf|headerImage)$") {
      # No point in compressing these
      remove req.http.Accept-Encoding;
    } elsif (req.http.Accept-Encoding ~ "gzip") {
      set req.http.Accept-Encoding = "gzip";
    } elsif (req.http.Accept-Encoding ~ "deflate") {
      set req.http.Accept-Encoding = "deflate";
    } else {
      # unkown algorithm
      remove req.http.Accept-Encoding;
    }
  }

  # Strip hash, server doesn't need it.
  if (req.url ~ "\#") {
    set req.url=regsub(req.url,"\#.*$","");
  }
  # Strip out Google related parameters
  if(req.url ~ "(\?|&)(utm_source|utm_medium|utm_campaign|gclid|cx|ie|cof|siteurl)=") {
    set req.url=regsuball(req.url,"&(utm_source|utm_medium|utm_campaign|gclid|cx|ie|cof|siteurl)=([A-z0-9_\-\.%25]+)","");
    set req.url=regsuball(req.url,"\?(utm_source|utm_medium|utm_campaign|gclid|cx|ie|cof|siteurl)=([A-z0-9_\-\.%25]+)","?");
    set req.url=regsub(req.url,"\?&","?");
    set req.url=regsub(req.url,"\?$","");
  }
/* End modifying the request */

  if (req.request == "PURGE") {
    if (!client.ip ~ purge) {
       error 405 "Not allowed.";
    }
    return(lookup);
  }

/* grace and saint related settings.
   To ensure to always serve static content. */
  if (!req.backend.healthy) {
    set req.grace = 1d;
  } else {
     set req.grace = 15m;
  }
/* end saint/grace mode stuff */

  return (lookup);
}

sub vcl_error {
  /* set a marker on so we know there is an error with the backends
     and that we should serve out stale content */
  if ( req.http.X-Varnish-Error != "1" && req.request != "PURGE" && req.restarts == 0) {
    set req.http.X-Varnish-Error = "1";
    return (restart);
  }
}

sub vcl_hash {
   set req.hash += req.url;
   if (req.http.Accept-Encoding) { set req.hash += req.http.Accept-Encoding; }
   return (hash);
}

sub vcl_fetch {
  unset beresp.http.set-cookie;
  if (beresp.status == 500) {
    set beresp.saintmode = 5s;
    set req.http.X-Varnish-Error = "1";
    return (restart);
  }

  /* override ttls */
  if(beresp.status == 301 || beresp.status == 302){
    /* all redirects can be cached for a long time. Granted we always have invalidation. */
    set beresp.ttl = 5h;
  } else if(req.url ~ ".*portal_css.+cachekey.*\.(css|js)$") {
    /* generated css/js files should be cached for a LONG time. All unique urls. */
    set beresp.ttl = 10d;
  } else if (req.url ~ "(\.jpg|\.png|\.gif|\.gz|\.tgz|\.bz2|\.tbz|\.mp3|\.ogg|\.pdf|\.css|\.js|/image_(large|preview|mini|thumb|tile))$") {
    /* all file type resources can be cached for an hour */
    set beresp.ttl = 1h;
  }else{
    /* everything else */
    set beresp.ttl = 30m; /* how long should varnish cache it? */
  }
  set beresp.grace = 10d; /* The max amount of time to keep object in cache */
  set beresp.http.X-Varnish-beresp-ttl = beresp.ttl;
  set beresp.http.X-Varnish-beresp-grace = beresp.grace;
  set beresp.http.X-Varnish-beresp-status = beresp.status;
}

sub vcl_hit {
   if (req.request == "PURGE") {
     set obj.ttl = 0s;
     error 200 "Purged.";
    }
}

sub vcl_miss {
  if (req.request == "PURGE") {
    error 404 "Not in cache.";
  }
}


sub vcl_deliver {
  if (obj.hits > 0) {
    set resp.http.X-Cache = "HIT";
  } else {
    set resp.http.X-Cache = "MISS";
  }
}

Additional Tips

  • Varnish doesn't have nice error messages, so use nginx to override 500 errors to your liking if, for some reason, there is an error on a resource that was not in the stale cache.
  • Varnish's cache is NOT persistent(although, varnish 3.0 is supposed to be) so if you restart your varnish process, you'll lose your long term cache.
  • Also, you're limited by the size of your size. If you have a large site, make sure that you set the varnish file cache size to something very large so that you're able to utilize the use of stale content.
Post

Plone 3.3.5 on Mac OS X Lion

Some tips for Plone 3 on Lion.

Getting Python 2.4

First off, make sure you have a version of python 2.4 installed on the system. If you use the one located in the svn collective, it has a few patches that make it work correctly with Lion.

svn co http://svn.plone.org/svn/collective/buildout/python/
cd python
python bootstrap.py
./bin/buildout 

Then use that python with your buildout.

Beware of collective.xdv

I didn't have enough time to figure out why, but xdv was making my instance crash on startup with no explanation. I did get plone to startup by upgrading to the latest version of xdv and collective.xdv but it would still crash when rendering a page for me. For now, I've just disable xdv on Lion and at least for a working plone 3 dev machine.

Small post but I just wanted to put it up in case someone else was experiencing the same problems.

Post

Fixing Broken ZODB Object references

I'm not an expert on this by any means, but here are some notes on my latest episode.

Introduction

If you start seeing POSKeyErrors on certain object, it most likely means your database is in some form of inconsistency. The problem is very well described by Elizabeth Leddy on her blog here. Her blog didn't quite handle the case that I encountered, missing objects--no oid in ZODB.

Getting Started

Run fsrefs.py to test your database and have it tell you which objects are bad.

python /path/to/eggs/ZODB/scripts/fsrefs.py /path/to/zodb/Data.fs

Will yield results like:

 oid 0x959755L BTrees.OOBTree.OOBucket
last updated: 2011-04-15 13:31:28.380634, tid=0x38DA88B79173877L
refers to invalid object:
	oid 0x0135ca66 missing: ''

oid 0x135CA59L Products.ATContentTypes.content.document.ATDocument
last updated: 2011-04-11 22:21:16.544874, tid=0x38D941D46976A11L
refers to invalid objects:
	oid 0x0135ca65 missing: ''
	oid 0x0135ca5c missing: ''

oid 0x135CA6AL BTrees.OOBTree.OOBTree
last updated: 2011-04-11 22:16:14.294142, tid=0x38D94183CFD03CCL
refers to invalid object:
	oid 0x0135ca6b missing: ''

Testing Out The Bad Object

from ZODB.utils import p64
from persistent import Persistent
obj = app._p_jar[p64(0x959755L)] obj

Should give the error:

2011-05-24 09:23:31 ERROR ZODB.Connection Couldn't load state for 0x0135ca59
Traceback (most recent call last):
  File "/opt/Zope/buildout-cache/eggs/ZODB3-3.8.4wc1-py2.4-linux-x86_64.egg/ZODB/Connection.py", line 811, in setstate
    self._setstate(obj)
  File "/opt/Zope/buildout-cache/eggs/ZODB3-3.8.4wc1-py2.4-linux-x86_64.egg/ZODB/Connection.py", line 870, in _setstate
    self._reader.setGhostState(obj, p)
  File "/opt/Zope/buildout-cache/eggs/ZODB3-3.8.4wc1-py2.4-linux-x86_64.egg/ZODB/serialize.py", line 604, in setGhostState
    state = self.getState(pickle)
  File "/opt/Zope/buildout-cache/eggs/ZODB3-3.8.4wc1-py2.4-linux-x86_64.egg/ZODB/serialize.py", line 597, in getState
    return unpickler.load()
  File "/opt/Zope/buildout-cache/eggs/ZODB3-3.8.4wc1-py2.4-linux-x86_64.egg/ZODB/serialize.py", line 471, in _persistent_load
    return self.load_oid(reference)
  File "/opt/Zope/buildout-cache/eggs/ZODB3-3.8.4wc1-py2.4-linux-x86_64.egg/ZODB/serialize.py", line 537, in load_oid
    return self._conn.get(oid)
  File "/opt/Zope/buildout-cache/eggs/ZODB3-3.8.4wc1-py2.4-linux-x86_64.egg/ZODB/Connection.py", line 244, in get
    p, serial = self._storage.load(oid, self._version)
  File "/opt/Zope/buildout-cache/eggs/ZODB3-3.8.4wc1-py2.4-linux-x86_64.egg/ZEO/ClientStorage.py", line 712, in load
    return self.loadEx(oid, version)[:2]
  File "/opt/Zope/buildout-cache/eggs/ZODB3-3.8.4wc1-py2.4-linux-x86_64.egg/ZEO/ClientStorage.py", line 735, in loadEx
    data, tid, ver = self._server.loadEx(oid, version)
  File "/opt/Zope/buildout-cache/eggs/ZODB3-3.8.4wc1-py2.4-linux-x86_64.egg/ZEO/ServerStub.py", line 196, in loadEx
    return self.rpc.call("loadEx", oid, version)
  File "/opt/Zope/buildout-cache/eggs/ZODB3-3.8.4wc1-py2.4-linux-x86_64.egg/ZEO/zrpc/connection.py", line 699, in call
    raise inst # error raised by server
POSKeyError: 0x0135ca65

Post

Notes on a More Secure Plone Deployment

Some things to think about if you're planning on providing a more secure Plone site. While Plone is a very secure CMS with an incredible track record, there are still plenty of things you can do to protect sites that might be larger targets.

Read-only Public Site

Making your public site read-only will prevent even a compromised site from taking any damage--even if a malicious user does somehow gain access, they can't save any different data to the database.

There are a few ways to do this:

  • Zope Replication Services(ZRS) allow you replicate a read-write backend private server to a read-only public facing site
  • You can also use RelStorage for you zeoserver. Then use the replication facilities provided by some RDMSs to replicate to a read-only zeoserver on the public site.
  • It is also possible to have read-only zeo clients connected to a read-write zeo server.
  • zeoraid might even be an option(never tried it)

One thing to note is that there are some cases where Plone will try to write on read unfortunately. To get around this, I create a before commit event handler in a policy product to abort every transaction when the server is read-only. It's kind of hackish but a necessary evil to prevent a user from getting a nasty ReadOnly database error thrown at them. It would look something like:

from zope.component import adapter
from ZPublisher.interfaces import IPubBeforeCommit
import App.config
import transaction
configuration = App.config.getConfiguration()
readonly = configuration.read_only_database
@adapter(IPubBeforeCommit)
def abortTransactionOnReadOnly(event):
if readonly:
transaction.abort()

Rewrite Login URLs

You can also rewrite login urls on the public site to restrict anyone from seeing a login form. Just do normal rewrites at your proxy server.

Urls you'll want to rewrite are:

  • /manage
  • /login
  • /logged_out
  • /require_login
  • /acl_users

This will prevent anyone from seeing a login form and an unauthorized page.

Post

Using Plone as a Document Repository

Sharing my experience in using plone to OCR PDF documents and displaying the documents in the browser with Flex Paper.

Update

It is recommended that you do not use this method anymore. Please use collective.documentviewer now which should cover all the use cases.

We just released a new site that houses thousands of scanned PDF documents that are now viewable in the browser via Flex Paper. We started with PDFs that were just scanned images. Plone, with the help of a few packages, then OCR'd and replaced the PDF with a searchable PDF counterpart.

Features

  • Convert Image PDFs to searchable versions
  • Split large PDFs into multiple documents
  • Overwrite metadata of PDF
  • OCR text is then searchable via Plone search
  • Online viewable version
  • All document processing is done via asynchronous processes so adding documents is not slow
  • Can monitor conversion asynchronous processes

Requirements

  • wc.pageturner : For online viewable PDFs
  • wildcard.pdfpal : heavy lifting in PDF processing
  • plone.app.async : asynchronously process PDF documents
  • Tesseract > 3.0.1 system package
  • swftools system package
  • ghostscript system package
  • hocr2pdf system package
  • pdftk system package
  • tiff2pdf system package

Caveats

  • Probably only works in Linux
  • wildcard.pdfpal is pretty specific and isn't smart at if it should process the PDF. For instance, if the PDF is already searchable, it'll still try to convert it regardless.
  • We're not really interested in wildly supporting pdfpal beyond our use case(that's why it's not listed on plone.org, but in the collective and on pypi). So if you're interested in implementing this, you might end up contributing to the project and cleaning up some of the cruft in the package.
Post

Running Plone 4b4 with Zope 2.13.0a1

Just some guidelines on getting Plone 4b4 to work with the new Zope 2.13 release to save you some time.

Update

This method is most likely no longer suitable for a plone WSGI setup. With stable releases of Zope 2.13 coming out and Plone 4.1 almost ready for alpha, it'd be best to start there and ignore this post.

Introductions

I'm not going to go into detail of the wheres and hows everything is done. This post expects you to know a bit about plone, zope and buildout. Maybe I'll be more detailed later. Use this if you want to save yourself a lot of time in getting a working setup with Plone 4 and Zope 2.13. Zope 2.13 adds native WSGI support in Zope. I tested it a bit and seems to work well but results may vary and I'm sure there will be a more supported way to do this soon.

Extends

Make sure your buildout extends the http://download.zope.org/Zope2/index/2.13.0a1/versions.cfg versions file.

Checkouts

You'll need to checkout Products.CMFCore, Products.PluggableAuthService, Products.TinyMCE and plone.locking from svn. You can use these locations right now until there is a new release,

http://svn.zope.org/repos/main/Products.CMFCore/branches/2.2/ Products.CMFCore

http://svn.zope.org/repos/main/Products.PluggableAuthService/trunk Products.PluggableAuthService

http://svn.plone.org/svn/collective/Products.TinyMCE/trunk/ Products.TinyMCE

http://svn.plone.org/svn/plone/plone.locking/trunk/ plone.locking

You'll also need to add these packages to your develop buildout section.

Extra Versions Pins

You'll need to pin these versions since Zope 2.13 doesn't pin version that plone's setup used to assume were pinned,

Add these extra version pins

Products.CMFCore = Products.PluggableAuthService = Products.TinyMCE = plone.locking = five.formlib = 1.0.2 zope.formlib = 3.7.0 zope.app.apidoc = 3.6.2 zope.app.applicationcontrol = 3.5.0 zope.app.appsetup = 3.11 zope.app.authentication = 3.6.0 zope.app.basicskin = 3.4.1 zope.app.broken = 3.5.0 zope.app.cache = 3.6.0 zope.app.catalog = 3.8.0 zope.app.component = 3.8.3 zope.app.container = 3.8.0 zope.app.content = 3.4.0 zope.app.dav = 3.5.1 zope.app.debug = 3.4.1 zope.app.dependable = 3.4.0 zope.app.dtmlpage = 3.5.0 zope.app.error = 3.5.2 zope.app.exception = 3.5.0 zope.app.file = 3.5.0 zope.app.folder = 3.5.1 zope.app.form = 3.8.1 zope.app.generations = 3.5.0 zope.app.http = 3.6.0 zope.app.i18n = 3.6.1 zope.app.interface = 3.5.0 zope.app.intid = 3.7.0 zope.app.locales = 3.6.1 zope.app.localpermission = 3.7.2 zope.app.pagetemplate = 3.7.1 zope.app.principalannotation = 3.7.0 zope.app.publication = 3.8.1 zope.app.publisher = 3.8.4 zope.app.renderer = 3.5.1 zope.app.rotterdam = 3.5.0 zope.app.schema = 3.5.0 zope.app.security = 3.7.3 zope.app.securitypolicy = 3.5.1 zope.app.server = 3.4.2 zope.app.session = 3.6.1 zope.app.testing = 3.7.3 zope.app.traversing = 3.4.0 zope.app.undo = 3.5.0 zope.app.wsgi = 3.6.0 zope.app.zapi = 3.4.1 zope.app.zcmlfiles = 3.5.5 zope.app.zopeappgenerations = 3.5.0 zope.app.zptpage = 3.5.0 plone.app.form = 2.0b6 plone.app.contentrules = 2.0b4 plone.app.portlets = 2.0b11 plone.app.users = 1.0b9 plone.app.contentmenu = 2.0b3

Extra Eggs

You'll also need to add extra add dependencies to your buildout that the Plone 4b4 egg doesn't require and should.

    zope.formlib

    five.formlib

    zope.app.schema


WSGI

You can take a look at my previous post for doing WSGI Zope2 for guidelines on how to setup the paste config and such.

Post

Running Plone 4 with a Zope2 WSGI

Guide to running Plone 4 with the Zope2 WSGI branch

Update

Tres has managed to merge his WSGI branch into trunk and Hanno tells me the unofficial plan is to include this in a release for Zope 2.13, in time for Plone 4.1. This is not decided upon yet though.

Overview

I was planning on implementing WSGI for Zope2 during the Penn State Symposium Sprints, and I did a few things to help out; however, Tres Seaver did most of the work on his own before and during part of the Sprint :) The rest of the time I spent just testing it out and helping with the Theme Editor sprint.

Now, it's really quite trivial to get it working now and makes all of the repoze.zope2 nonsense unneeded now. This article is just here for a reference if anyone else is interested in getting it going on their setup.

Guide

This guide assumes you have an existing Plone 4 installation to work from. I don't provide any buildouts here--just modifying an existing buildout to make it work with a branch of Zope2 and creating an ini that Paste can consume to serve WSGI.

Supplying the WSGI'd Zope2

First off, go to the src directory of the installation. If you've installed using the unified installer, that will bin in instance-home/zinstance/src or if you just used straight buildout, it'll be in instance/src. Then checkout the Zope2 branch:

svn co http://svn.zope.org/repos/main/Zope/branches/tseaver-fix_wsgi/ Zope2

Stringing up buildout

Next thing you'll need to do is modify your buildout.cfg file to add the checked out Zope 2 to the develop section:

develop = 
...
Zope2
...

Still modifying your buildout.cfg, add Paste, PasteScript, repoze.tm2 and repoze.retry to your eggs section:

eggs =
Plone
Paste
PasteScript
repoze.tm2
repoze.retry 

Again, editing your buildout.cfg, add a paster part for the paster script:

parts =
...
paster
...

[paster]
recipe = repoze.recipe.egg
scripts = paster
eggs = ${instance:eggs}

Then, you'll need to add the updated Zope2 versions for the WSGI branch. To do this, basically, just add the versions.cfg file provided in the branch after every other version file listed in the extends directive. It'll look like this:

extends =
...
src/Zope2/versions.cfg

Then run your buildout like normal:

./bin/buildout

Creating a WSGI Configuration File

You'll now need to create a WSGI configuration file. Right now, we'll just server it using the Paste server and wsgi ini configuration way. You can also do this to string up Apache's WSGI implementation but that is beyond the scope of this article.

Create a file in the instance directory called, zope2.ini with the contents of:

[app:zope]
use = egg:Zope2#main
zope_conf = %(here)s/parts/instance/etc/zope.conf

[pipeline:main]
pipeline =
    egg:paste#evalerror
    egg:repoze.retry#retry
    egg:repoze.tm2#tm
    zope

[server:main]
use = egg:paste#http
host = localhost
port = 8080

The zope_conf value in the app:zope section can be the path to any zope.conf file. I'm just exampling the standard location of it and not going through the configuration of that file itself.

Fire it up!

If all went well, you should now be able to start up your Plone 4 instance on WSGI like this:

./bin/paster serve zope2.ini

Your server should now be able to visit your site on http://localhost:8080

Caveats

I did run into a snag with the Mac OS X unified installer and the version of python it has configured. Basically, it wouldn't compile the Zope2 dependencies so I had to use my own version of python that I had compiled with the python buildout found in the plone collective svn. The bug is sort of referenced in the zope bug tracker.

Future Considerations

I'm hoping to maybe get a release with this branch implementation out--maybe as an alpha or beta release since I don't think they are planning on merging this to core any time soon; although, I really have no understanding of what that whole process is.

I'd like to see the Zope2 package implement mkzope2instance and other convenience methods so it'd be possible to install Plone 4/Zope2 without buildout at all maybe using pip. I'm looking into how this might be able to happen with a pip versions file and other things. Maybe more on this later.

 

Post any comments if you run into any issues.

Post

Plone 4 Upgrade

Thoughts on moving the server to Plone 4.

Last night I decided to take some time to move this site over to Plone 4. The repoze.zope2 build I had previously was buggy and I am starting to dislike Deliverance a bit.

Snags

  • the search didn't work because of this migration issue http://dev.plone.org/plone/ticket/10360
  • old file types aren't migrated to blob http://dev.plone.org/plone/ticket/10365
  • archetypes don't want to use the new add views--should probably report this
  • If you have the jquery UI package installed, it'll mess with some of Plone's JS for some reason--I just disabled it.
  • some other control panel entries and menu items were not fully migrated, still pointing to old templates

 

Benefits

  • FAST--running this on a linode with 512MB of ram. 2 zeo clients and with CacheSetup, this thing is very fast. Much faster than Plone 3.
  • Just more polished than Plone 3--with usability enhancements and a new theme, it's great.
  • I haven't found a product that wasn't compatible with Plone 4 yet--although, sometimes I needed to do some digging or use an svn checkout to get the compatible version

 

comments powered by Disqus

Navigation