Author Archives: funcptr

Property class on AppEngine does not support a tuple for choices

Posted on by

Google AppEngine does not allow one to set the choices field of a model property to a tuple; this workaround will have have a form give the user the choices with the user friendly items but have the backend store the non-user friendly version. 

# Set up our non-user friendly and user friendly choices
REDIRECT_CHOICES = (
    (301, "301 - Permanent Redirect"),
    (302, "302 - Temporary Redirect"),
    (403, "403 - Access Denied"),
    (404, "404 - File Not Found"),
    (500, "500 - Internal Server Error")
)

# Sample model that has the required Property with the choices set to the non-user friendly values
class Shorturl(db.Model):
    uripath = db.StringProperty(verbose_name="Path", required=True)
    httpcode = db.IntegerProperty(verbose_name="HTTP code", required=True, default=301, choices=[x[0] for x in REDIRECT_CHOICES])
    location = db.StringProperty(verbose_name="Location")

# This is our djangoform that has the choices set as required so that the user gets the user friendly version and not just simply
# the numbers.
class ShorturlForm(djangoforms.ModelForm):
    httpcode = forms.IntegerField(widget=forms.Select(choices=REDIRECT_CHOICES), required=True, label='HTTP code')

    class Meta:
        model = Shorturl

Given the above tuple it will set the choices in the model for Google AppEngine and pass it to the field on the Django form allowing user friendly output. This will allow the user to pick the right redirect choice without having to know that various different redirect codes or have to look them up.

Upgrading an OpenSolaris ipkg zone

Posted on by

Recently I was working on my OpenSolaris machine (file server, ZFS rocks, will write more on that later), and I have one non-global zone, one that use for web development, which is aptly called dev-web.

So the following shows up when I run zoneadm:

xistence@Keyhole.network.lan:~# zoneadm list -iv
  ID NAME             STATUS     PATH                           BRAND    IP    
   0 global           running    /                              native   shared
   4 dev-web          running    /storage/zones/dev-web         ipkg     shared

The thing is, I had upgraded the global zone with the latest update available for the version (snv_101b):

pkg image-update -v

This had not upgraded my one none-global zone. And running pkg image-update from within the zone itself is not possible, because you can't do an upgrade on a "live" system, mainly because the image-update wants to create a new bootable environment, something that was already created when I upgraded the global zone. So what we have to do is mount the non-global zone to /mnt and tell pkg with -R where to find it and upgrade it anyway!

First we are going to halt the current zone, since I am mean and nothing important is running on the test bed system, I just used:

zoneadm -z dev-web halt

However, the better way is to off course use the shutdown command:

zlogin dev-web shutdown

and then for good measure a halt!

Next up, looking at what we are going to be mounting.

zfs list
NAME                               USED  AVAIL  REFER  MOUNTPOINT
rpool                             11.9G  42.7G    75K  /rpool
rpool/ROOT                        2.93G  42.7G    18K  legacy
rpool/ROOT/opensolaris            7.34M  42.7G  2.74G  /
rpool/ROOT/opensolaris-1          2.92G  42.7G  2.74G  /
rpool/dump                        1.87G  42.7G  1.87G  -
rpool/export                      5.27G  42.7G    19K  /export
rpool/export/home                 5.27G  42.7G    21K  /export/home
rpool/export/home/guest           5.24G  14.8G  5.24G  /export/home/guest
rpool/export/home/xistence        37.0M  42.7G  37.0M  /export/home/xistence
rpool/swap                        1.87G  43.0G  1.51G  -
storage                            493G  3.08T  35.1K  /storage
storage/media                      398G  3.08T   398G  /storage/media
storage/virtualbox                2.10G  3.08T  2.10G  /storage/virtualbox
storage/xistence                  91.7G  3.08T  91.7G  /storage/xistence
storage/zones                      957M  99.1G  30.4K  /storage/zones
storage/zones/dev-web              957M  19.1G  32.0K  /storage/zones/dev-web
storage/zones/dev-web/ROOT         957M  19.1G  28.8K  legacy
storage/zones/dev-web/ROOT/zbe    1.60M  19.1G   936M  legacy
storage/zones/dev-web/ROOT/zbe-1   955M  19.1G   935M  legacy

When pkg image-update was run on the global zone it created a new bootable environment named opensolaris-1, the cool thing is, that beadm at the same time will also create a new bootable environment for your zones. That way you can upgrade your zones afterwards, and if stuff does not work, you can revert the ENTIRE machine back to the previous state (ZFS is cool like that), thereby also making sure that your zones are reverted so that there are no incompatibilities.

So what we are looking for in this case is a zbe-1, this is the new root for the zone that we need to update, so we now need to mount it.

mount -F zfs storage/zones/dev-web/ROOT/zbe-1 /mnt

Note, that there is no / in front of storage, this is because we are specifying a pool name, since there is no "real" path that is defined as /storage/zones/dev-web/ROOT/zbe-1. Now that it is mounted, we are able to pass the -R flag to pkg, to get it to update our zone:

xistence@Keyhole.network.lan:~# pkg -R /mnt image-update -v
Creating Plan / Before evaluation:     
UNEVALUATED:
+pkg:/entire@0.5.11,5.11-0.101:20081204T010954Z

After evaluation:
pkg:/entire@0.5.11,5.11-0.101:20081119T235706Z -> pkg:/entire@0.5.11,5.11-0.101:20081204T010954Z
Actuators:

None
PHASE                                        ACTIONS
Update Phase                                     1/1 
PHASE                                          ITEMS
Reading Existing Index                           9/9 
Indexing Packages                                1/1

---------------------------------------------------------------------------
NOTE: Please review release notes posted at:
   http://opensolaris.org/os/project/indiana/resources/relnotes/200811/x86/
---------------------------------------------------------------------------

Voilá, and the deed is done. The last command is to off course unmount the zone, that we can then issue a zoneadm boot command to start it back up:

umount /mnt
zoneadm -z dev-web boot

And then on the zone after we log into it (over SSH in my case):

xistence@webdev.network.lan:~# pkg list -u
No installed packages have available updates

Which is exactly what we wanted! Your zone is now upgraded with the latest version available from the global zone.

Upgrading an OpenSolaris ipkg zone

Posted on by

Recently I was working on my OpenSolaris machine (file server, ZFS rocks, will write more on that later), and I have one non-global zone, one that use for web development, which is aptly called dev-web.

So the following shows up when I run zoneadm:

xistence@Keyhole.network.lan:~# zoneadm list -iv
  ID NAME             STATUS     PATH                           BRAND    IP    
   0 global           running    /                              native   shared
   4 dev-web          running    /storage/zones/dev-web         ipkg     shared

The thing is, I had upgraded the global zone with the latest update available for the version (snv_101b):

pkg image-update -v

This had not upgraded my one none-global zone. And running pkg image-update from within the zone itself is not possible, because you can't do an upgrade on a "live" system, mainly because the image-update wants to create a new bootable environment, something that was already created when I upgraded the global zone. So what we have to do is mount the non-global zone to /mnt and tell pkg with -R where to find it and upgrade it anyway!

First we are going to halt the current zone, since I am mean and nothing important is running on the test bed system, I just used:

zoneadm -z dev-web halt

However, the better way is to off course use the shutdown command:

zlogin dev-web shutdown

and then for good measure a halt!

Next up, looking at what we are going to be mounting.

zfs list
NAME                               USED  AVAIL  REFER  MOUNTPOINT
rpool                             11.9G  42.7G    75K  /rpool
rpool/ROOT                        2.93G  42.7G    18K  legacy
rpool/ROOT/opensolaris            7.34M  42.7G  2.74G  /
rpool/ROOT/opensolaris-1          2.92G  42.7G  2.74G  /
rpool/dump                        1.87G  42.7G  1.87G  -
rpool/export                      5.27G  42.7G    19K  /export
rpool/export/home                 5.27G  42.7G    21K  /export/home
rpool/export/home/guest           5.24G  14.8G  5.24G  /export/home/guest
rpool/export/home/xistence        37.0M  42.7G  37.0M  /export/home/xistence
rpool/swap                        1.87G  43.0G  1.51G  -
storage                            493G  3.08T  35.1K  /storage
storage/media                      398G  3.08T   398G  /storage/media
storage/virtualbox                2.10G  3.08T  2.10G  /storage/virtualbox
storage/xistence                  91.7G  3.08T  91.7G  /storage/xistence
storage/zones                      957M  99.1G  30.4K  /storage/zones
storage/zones/dev-web              957M  19.1G  32.0K  /storage/zones/dev-web
storage/zones/dev-web/ROOT         957M  19.1G  28.8K  legacy
storage/zones/dev-web/ROOT/zbe    1.60M  19.1G   936M  legacy
storage/zones/dev-web/ROOT/zbe-1   955M  19.1G   935M  legacy

When pkg image-update was run on the global zone it created a new bootable environment named opensolaris-1, the cool thing is, that beadm at the same time will also create a new bootable environment for your zones. That way you can upgrade your zones afterwards, and if stuff does not work, you can revert the ENTIRE machine back to the previous state (ZFS is cool like that), thereby also making sure that your zones are reverted so that there are no incompatibilities.

So what we are looking for in this case is a zbe-1, this is the new root for the zone that we need to update, so we now need to mount it.

mount -F zfs storage/zones/dev-web/ROOT/zbe-1 /mnt

Note, that there is no / in front of storage, this is because we are specifying a pool name, since there is no "real" path that is defined as /storage/zones/dev-web/ROOT/zbe-1. Now that it is mounted, we are able to pass the -R flag to pkg, to get it to update our zone:

xistence@Keyhole.network.lan:~# pkg -R /mnt image-update -v
Creating Plan / Before evaluation:     
UNEVALUATED:
+pkg:/entire@0.5.11,5.11-0.101:20081204T010954Z

After evaluation:
pkg:/entire@0.5.11,5.11-0.101:20081119T235706Z -> pkg:/entire@0.5.11,5.11-0.101:20081204T010954Z
Actuators:

None
PHASE                                        ACTIONS
Update Phase                                     1/1 
PHASE                                          ITEMS
Reading Existing Index                           9/9 
Indexing Packages                                1/1

---------------------------------------------------------------------------
NOTE: Please review release notes posted at:
   http://opensolaris.org/os/project/indiana/resources/relnotes/200811/x86/
---------------------------------------------------------------------------

Voilá, and the deed is done. The last command is to off course unmount the zone, that we can then issue a zoneadm boot command to start it back up:

umount /mnt
zoneadm -z dev-web boot

And then on the zone after we log into it (over SSH in my case):

xistence@webdev.network.lan:~# pkg list -u
No installed packages have available updates

Which is exactly what we wanted! Your zone is now upgraded with the latest version available from the global zone.

Using DJB’s daemontools and netcat to bounce a request around

Posted on by

Lately I have been moving the data from one server to another server, located halfway across the globe, and I needed some way to forward all incoming requests from the old server to the new server. This had to be done so that DNS could take its time to update while everything was now already being served from the new location.

What I ended up doing was using daemontools along with netcat to pipe the request around the world. Here are the steps I took in doing so:

mkdir /usr/local/redirect/
cd /usr/local/redirect/
mkdir smtp smtp/env
cd smtp

Put the following in a file named run:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
#!/bin/sh
exec 2>&1 \
envdir ./env \
sh -c '
    case "$REMOTENAME" in h) H=;; p) H=p;; *) H=H;; esac
    case "$REMOTEINFO" in r) R=;; [0-9]*) R="t$REMOTEINFO";; *) R=R;; esac
    exec \
    /usr/local/bin/tcpserver \
        -vDU"$H$R" \
        ${LOCALNAME+"-l$LOCALNAME"} \
        ${BACKLOG+"-b$BACKLOG"} \
        ${CONCURRENCY+"-c$CONCURRENCY"} \
        -- "${IP-0}" "${PORT}" \
        /usr/bin/nc "${REMOTEHOST}" "${REMOTEPORT}"
'

Make it executable:

chmod +x run

Then we need to set up a few environment variables:

cd env
echo "example.net" > REMOTEHOST
echo "25" > REMOTEPORT
echo "25" > PORT
echo `hostname` > LOCALNAME
echo "200" > CONCURRENCY

I made REMOTEPORT and PORT be separate on purpose, in one case I now had more IP's than before, so instead of having SSL running on a separate port it was running on the default port, and I needed a clean way to forward that.

Now just add it to your services folder as a symlink and it will automatically be started, from there it will do it's job! I also suggest adding some simple logging, or discarding all off the output from tcpserver.

Using DJB’s daemontools and netcat to bounce a request around

Posted on by

Lately I have been moving the data from one server to another server, located halfway across the globe, and I needed some way to forward all incoming requests from the old server to the new server. This had to be done so that DNS could take its time to update while everything was now already being served from the new location.

What I ended up doing was using daemontools along with netcat to pipe the request around the world. Here are the steps I took in doing so:

mkdir /usr/local/redirect/
cd /usr/local/redirect/
mkdir smtp smtp/env
cd smtp

Put the following in a file named run:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
#!/bin/sh
exec 2>&1 \
envdir ./env \
sh -c '
    case "$REMOTENAME" in h) H=;; p) H=p;; *) H=H;; esac
    case "$REMOTEINFO" in r) R=;; [0-9]*) R="t$REMOTEINFO";; *) R=R;; esac
    exec \
    /usr/local/bin/tcpserver \
        -vDU"$H$R" \
        ${LOCALNAME+"-l$LOCALNAME"} \
        ${BACKLOG+"-b$BACKLOG"} \
        ${CONCURRENCY+"-c$CONCURRENCY"} \
        -- "${IP-0}" "${PORT}" \
        /usr/bin/nc "${REMOTEHOST}" "${REMOTEPORT}"
'

Make it executable:

chmod +x run

Then we need to set up a few environment variables:

cd env
echo "example.net" > REMOTEHOST
echo "25" > REMOTEPORT
echo "25" > PORT
echo `hostname` > LOCALNAME
echo "200" > CONCURRENCY

I made REMOTEPORT and PORT be separate on purpose, in one case I now had more IP's than before, so instead of having SSL running on a separate port it was running on the default port, and I needed a clean way to forward that.

Now just add it to your services folder as a symlink and it will automatically be started, from there it will do it's job! I also suggest adding some simple logging, or discarding all off the output from tcpserver.