Infinite | Squares

Art of code and more

Install Open edX on Ubuntu 12.04 at Digital Ocean

Open edX is the open source platform that powers edX courses. It is freely available to the community. Institutions can host their own instances of Open edX and offer their own classes. Educators can extend the platform to build learning tools that precisely meet their needs. And developers can contribute new features to the Open edX platform. There are already many online course sites developed using Open edX.

Architecture

Open edX consisting of several web applications, mostly written with Django Python. There are also mobile app for Android and iOS and analytics platform powered by Hadoop. It is quite complete and we can deploy it to have a complete online course experience. The architecture is explained here.

Open edX has several components :

  • LMS (App visible to students)
  • CMS (Studio. App visible to course creator)
  • cs_comment_service (Discussions)
  • MySQL (Database for user data and others)
  • Mongo (Database for course)
  • Mobile Apps (Android and iOS)
  • Analytics (Insight in edx.org)
  • Search with Elasticsearch
  • Background workers for various purposes (queued and distributed using Celery and RabbitMQ)

Installation

To start with Open edX, we need to install it first. There are several options for Open edX platform installation:

  • The Open edX Developer Stack, known as Devstack, is a Vagrant instance designed for local development.
  • Open edX Fullstack, known as Fullstack, has several deployment options:
    • Vagrant instance designed for deploying all edX services on a single server.
    • Open edX Fullstack on Ubuntu 12.04 64 bit

All installation methods mentioned above are meant for development purposes and not live production websites, since all of the components (apps and databases) are installed on single servers.

Since my laptop is not suitable for work with VirtualBox, I opted to install Open edX on Ubuntu server on cloud. For the cloud provider, I choose Digital Ocean. Digital Ocean provide SSD backed cloud server and its quite cheap. And this is my Open edX installation notes on Ubuntu 12.04 at Digital Ocean. Note that this is based on my experience as of today (2015-07-05).

Installation notes on Digital Ocean

Create a server

First, create Digital Ocean account if you don’t have one. You need to add your credit card so you can create your first droplet (server).

I choose droplet with 4 CPUs and 8 GB RAM ($80/mo). I think 2 CPUs and 4 GB RAM ($40/mo) could work but maybe installation will take a bit longer. Open edX has lots of components. Choose Image Ubuntu 12.04.5 x64.

I recommend to Add SSH keys for your Ubuntu server. It is considered more secure than password based access, and you don’t have to type password each time you access it. You can follow this tutorial to create SSH key for your server.

This is my droplet summary

  • Hostname : OpenEDX
  • Size : $80/mo
  • Region : Singapore 1
  • Image : 12.04.5 x64
  • Settings : none
  • SSH Keys : DigitalOcean

When you’re ready create your droplet. It will take about one minute.

Add swap space

Before we continue with Open edX installation. Lets create a swap space to add total amount of virtual memory allocated for our single server.

Again, folks and Digital Ocean has created a tutorial for creating swap space on Ubuntu server. Follow those steps with one exception, I want to create a 8 GB swap space and I use this command:

1
# sudo dd if=/dev/zero of=/swapfile bs=1G count=8

The tutorial page use 1024 bytes blocksize and it takes too long to create 8 GB swap. Plus, I am having hard time calculate the correct count (I experienced error that all the disk content is used as swap. Which is not what you want)

Install Open edX

After the server ready, now we can install Open edX. The install script is already available on Open edX repository on Github. We need to update the server first.

1
2
3
4
# sudo apt-get update -y
# sudo apt-get upgrade -y
# sudo apt-get install htop
# sudo reboot

I install htop so I can monitor what happens during installation. It takes a while and sometimes I wonder what is going on. Is it hang or doing something (compiling, downloading, etc)

Then run this command

1
# wget https://raw.githubusercontent.com/edx/configuration/master/util/install/sandbox.sh -O - | bash

It will run ansible commands to install all required dependencies and services for running Open edX on single server.

While we wait, lets read some Open edX documentation

Then I experiences series of errors:

Error 1
1
2
3
4
5
6
7
8
9
10
Error
  Downloading dm.xmlsec.binding-1.3.2.tar.gz (119kB)
    Error: cannot get XMLSec1 pre-processor and compiler flags; do you have the `libxmlsec1` development package installed?
    Complete output from command python setup.py egg_info:
    Error: cannot get XMLSec1 pre-processor and compiler flags; do you have the `libxmlsec1` development package installed?

    ----------------------------------------
    Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-RMvh3g/dm.xmlsec.binding

FATAL: all hosts have already failed -- aborting

The install script is missing one package: libxmlsec1-dev. Let’s install it

1
# sudo apt-get install libxmlsec1-dev

Redo the process by calling

1
# wget https://raw.githubusercontent.com/edx/configuration/master/util/install/sandbox.sh -O - | bash

Continue reading.

Error 2
1
2
3
4
5
6
Error
    swig -python -I/usr/include/python2.7 -I/usr/include/x86_64-linux-gnu -I/usr/include -I/usr/include/openssl -includeall -modern -o SWIG/_m2crypto_wrap.c SWIG/_m2crypto.i

    unable to execute swig: No such file or directory

    error: command 'swig' failed with exit status 1

The install script is again missing one package: swig.

1
# sudo apt-get install swig

Redo the process

1
# wget https://raw.githubusercontent.com/edx/configuration/master/util/install/sandbox.sh -O - | bash

And continue reading.

Error 3
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
...
- Migration 'student:0031_drop_student_anonymoususerid_temp_archive' is marked for no-dry-run.
 > student:0032_add_field_UserProfile_country_add_field_UserProfile_city
 > student:0032_auto__add_loginfailures
 > student:0033_auto__add_passwordhistory
 > student:0034_auto__add_courseaccessrole
 > student:0035_access_roles
 - Migration 'student:0035_access_roles' is marked for no-dry-run.
 ! Error found during real run of migration! Aborting.

 ! Since you have a database that does not support running
 ! schema-altering statements in transactions, we have had
 ! to leave it in an interim state between migrations.

! You *might* be able to recover with:   (migration cannot be dry-run; cannot discover commands)
 ! The South developers regret this has happened, and would
 ! like to gently persuade you to consider a slightly
 ! easier-to-deal-with DBMS (one that supports DDL transactions)
 ! NOTE: The error which caused the migration to fail is further up.
Error in migration: student:0035_access_roles
changed: [localhost] => (item=cms)

FATAL: all hosts have already failed -- aborting

This is error during database migration. I just redo the whole process.

You might wonder is it safe to redo the install process over and over again. It is safe because Ansible is designed to be idempotent which means that you can apply and re-apply it an infinite number of times without changing the result. I just assume the Open edX authors utilize idempotency in their Ansible playbooks.

Error 4
1
2
3
4
5
Error
failed: [localhost] => {"failed": true, "item": ""}
msg: Failed to validate the SSL certificate for download.elasticsearch.org:443. Use validate_certs=no or make sure your managed systems have a valid CA certificate installed. Paths checked for this platform: /etc/ssl/certs, /etc/pki/ca-trust/extracted/pem, /etc/pki/tls/certs, /usr/share/ca-certificates/cacert.org, /etc/ansible

FATAL: all hosts have already failed -- aborting

It means we cannot proceed to download ElasticSearch installer because our system installed certificates do not recognize SSL used in download.elasticsearch.org. One way to fix this, as suggested by the error message, add validate_certs=no somewhere when we install ElasticSearch. Open playbook for ElasticSearch:

1
# nano /var/tmp/configuration/playbooks/roles/elasticsearch/tasks/main.yml

At line 37, right after force=no, and press space to insert new line. Type validate_certs=no in a new line.

Then redo install using following script:

1
# cd /var/tmp/configuration/playbooks && sudo ansible-playbook -c local ./edx_sandbox.yml -i "localhost,"

Note that I redo Ansible process by executing local script instead of fetching again from github because we don’t want to lose our changes.

Successful install

After a while, finally there are no more errors on the installation process, and I have my Open edX running.

To make our life easier, I have forked edx configuration repo and you can execute it on Ubuntu 12.04 machine with

1
# wget https://cdn.rawgit.com/arifsetiawan/configuration/master/util/install/sandbox.sh -O - | bash

I am still learning Open edX and if you find any errors or have question or suggestion, please let me know.

Next, we will continue on exploring how settings work on our Open edX instance.

Comments