PDF Thumbnail Icon Preview Fix for Windows 10

If you have a big collection of books/documents in PDF format, it will be nice that you are able to see their preview thumbnails in File Explorer.  This had been a problem for 64 bit Windows 7 and 10 for a long time. PDF thumbnails are not showing in Windows Explorer. All you can see in file explorer are some generic Adobe icons and their boring file names. Some people even see horrible black square thumbnail icons.

no-preview-640

Apparently, that thumbnail preview functionality was deliberately removed in Acrobat Reader – There are an 8 page long discussion thread over the Adobe forum if you want to check it out (File preview thumbnails aren’t working for Acrobat DC files in Windows Explorer).

Third party fixes such as Leo Davidson’s Adobe PDF x64 fixes worked for me when I was using Windows 7. But after upgraded to Windows 10, nothing seems to work any more. Windows 10 fails to display any PDF thumbnails even with Davidson’s fixes installed. Fortunately, Adode has finally got its act together and fix this problem in their latest Acrobat Reader. All you need to do now is to update you Acrobat to the latest version – there is a constant reminder for you to update anyway, if you have not opted out of the automatic update notification.

acrobat_dc_preferences-600

Once that is done, open Acrobat, head over to Edit -> Preferences or just press Ctrl + K to open the Preferences window. Select General from the left hand side Categories menu. Then you will see near the middle of the screen Enable PDF thumbnail previews in Windows Explorer. Tick that box and hit OK then you are done. Hooray! Now I can see all my favourite MagPi magazines and pick out the one I want to read!

preview-enabled-640

Presumably this will work with Windows 7 too – I will confirm it in my Windows 7 VM later.

Changing the Index of a pandas DataFrame Copy Also Affects the Original

This was observed while I was trying play with the Index and MultiIndex of pandas.DataFrame objects. If I make a copy of a DataFrame and change one of the values of its index, the same index in the original DataFrame aslo changes.

In [1]:
from pandas import DataFrame
import pandas
import numpy as np
from IPython.display import HTML

pandas.__version__
Out[1]:
'0.17.1'

1. Let’s define functions to generate and display these DataFrames together

The functions below display the DataFrame side by side for ease of viewing.

In [2]:
def showDFs(dfs):
    html = ''
    for k, v in dfs.items():
        html += ('''<div style=\"float:left; padding:10px;\">
                 DataFrame: <strong>{}</strong>{}</div>'''
                 .format(k , v.to_html()))
    return HTML('<div style="float:left; width:100%;">' + html + '</div>')

def show():
    return showDFs({'df_original': df_original, 'df_copy':df_copy})

def make2frames():
    s = np.arange(20).reshape(4,5)
    d = DataFrame(s, index=list('abcd'), columns=list('ABCDE'))
    return d, d.copy()

2. Create DataFrames from a numpy array

Now let’s create a DataFrame then make a copy.

In [3]:
df_original, df_copy = make2frames()
show()
Out[3]:
DataFrame: df_original
A B C D E
a 0 1 2 3 4
b 5 6 7 8 9
c 10 11 12 13 14
d 15 16 17 18 19
DataFrame: df_copy
A B C D E
a 0 1 2 3 4
b 5 6 7 8 9
c 10 11 12 13 14
d 15 16 17 18 19

3. Make change to the index values of the DataFrames

Changes in the copy affects the original and vice versa.

In [4]:
df_copy.index.values[1] = 'NEW VALUE'
show()
Out[4]:
DataFrame: df_original
A B C D E
a 0 1 2 3 4
NEW VALUE 5 6 7 8 9
c 10 11 12 13 14
d 15 16 17 18 19
DataFrame: df_copy
A B C D E
a 0 1 2 3 4
NEW VALUE 5 6 7 8 9
c 10 11 12 13 14
d 15 16 17 18 19
In [5]:
df_original.index.values[3] = 'New Value 2'
show()
Out[5]:
DataFrame: df_original
A B C D E
a 0 1 2 3 4
NEW VALUE 5 6 7 8 9
c 10 11 12 13 14
New Value 2 15 16 17 18 19
DataFrame: df_copy
A B C D E
a 0 1 2 3 4
NEW VALUE 5 6 7 8 9
c 10 11 12 13 14
New Value 2 15 16 17 18 19

This is very much unexpected to me. I went on and tested the following changes to see whether I can find something else that I don’t expect just to confirm all that I have learn from pandas.

4. Making change to the values

As expected, making changes to the values in either DataFrame does not affect each other.

In [6]:
#reset both DataFrames
df_original, df_copy = make2frames()

# change the copy
df_copy['B'] = 1000
show()
Out[6]:
DataFrame: df_original
A B C D E
a 0 1 2 3 4
b 5 6 7 8 9
c 10 11 12 13 14
d 15 16 17 18 19
DataFrame: df_copy
A B C D E
a 0 1000 2 3 4
b 5 1000 7 8 9
c 10 1000 12 13 14
d 15 1000 17 18 19
In [7]:
#chage the original
df_original['C'] = 9999
show()
Out[7]:
DataFrame: df_original
A B C D E
a 0 1 9999 3 4
b 5 6 9999 8 9
c 10 11 9999 13 14
d 15 16 9999 18 19
DataFrame: df_copy
A B C D E
a 0 1000 2 3 4
b 5 1000 7 8 9
c 10 1000 12 13 14
d 15 1000 17 18 19

5. Making changes to column names

In [8]:
#reset both DataFrames
df_original, df_copy = make2frames()

#change the copy
df_copy.columns = list('FGHIJ')
show()
Out[8]:
DataFrame: df_original
A B C D E
a 0 1 2 3 4
b 5 6 7 8 9
c 10 11 12 13 14
d 15 16 17 18 19
DataFrame: df_copy
F G H I J
a 0 1 2 3 4
b 5 6 7 8 9
c 10 11 12 13 14
d 15 16 17 18 19
In [9]:
#change the original
df_original.columns = list('PQRST')
show()
Out[9]:
DataFrame: df_original
P Q R S T
a 0 1 2 3 4
b 5 6 7 8 9
c 10 11 12 13 14
d 15 16 17 18 19
DataFrame: df_copy
F G H I J
a 0 1 2 3 4
b 5 6 7 8 9
c 10 11 12 13 14
d 15 16 17 18 19

6. Making changes to the index – the ‘conventional way’

In [10]:
#reset both DataFrames
df_original, df_copy = make2frames()

df_copy.set_index([['k', 'l', 'm', 'n']], inplace=True)
show()
Out[10]:
DataFrame: df_original
A B C D E
a 0 1 2 3 4
b 5 6 7 8 9
c 10 11 12 13 14
d 15 16 17 18 19
DataFrame: df_copy
A B C D E
k 0 1 2 3 4
l 5 6 7 8 9
m 10 11 12 13 14
n 15 16 17 18 19
In [11]:
df_original.set_index([['one', 'two', 'three', 'four']], inplace=True)
show()
Out[11]:
DataFrame: df_original
A B C D E
one 0 1 2 3 4
two 5 6 7 8 9
three 10 11 12 13 14
four 15 16 17 18 19
DataFrame: df_copy
A B C D E
k 0 1 2 3 4
l 5 6 7 8 9
m 10 11 12 13 14
n 15 16 17 18 19

7. Now the indices are different in the two frames, let’s see if they are still affecting each other.

In [12]:
df_copy.index.values[1] = 'NEW VALUE'
show()
Out[12]:
DataFrame: df_original
A B C D E
one 0 1 2 3 4
two 5 6 7 8 9
three 10 11 12 13 14
four 15 16 17 18 19
DataFrame: df_copy
A B C D E
k 0 1 2 3 4
NEW VALUE 5 6 7 8 9
m 10 11 12 13 14
n 15 16 17 18 19
In [13]:
df_original.index.values[1] = 'ANOTHER VALUE'
show()
Out[13]:
DataFrame: df_original
A B C D E
one 0 1 2 3 4
ANOTHER VALUE 5 6 7 8 9
three 10 11 12 13 14
four 15 16 17 18 19
DataFrame: df_copy
A B C D E
k 0 1 2 3 4
NEW VALUE 5 6 7 8 9
m 10 11 12 13 14
n 15 16 17 18 19

8. Thoughts

In conclusion, when a fresh copy is made from a DataFrame, altering the individual values of either index with affect both the original and the copy. It is most likely that the index of the copied DataFrame is actually still pointing to the same memory location as the original until either of their reference is changed, i.e. they are pointing to different memory locations. When that happens, they are not going to affect each other any longer.

WordPress Blogging with Jupyter Notebook in Five Simple Steps

Jupyter Notebook is such a great tool for scientific programming and data analysis. I would like to be able to show some of my notebooks easily on this WordPress powered blog. After looking around the web and tweaking a little, I have figured out how to easily embed the contents of a notebook. In fact, this entire post was written in the notebook.

1. Writing the contents

First let’s try to use the notebook to do some analysis and plotting:

In [1]:
import re
from pandas import Series
import matplotlib.pyplot as plt
%matplotlib inline

Here we have a protein sequence of HLA-A*01. We are going to clean it up a little and plot the frequcies of each unique amino acid residue in the sequence:

In [2]:
HLA_A1 ='''MAVMAPRTLLLLLSGALALTQTWAGSHSMRYFFTSVSRPGRGEPRFIAVGYVDDTQFVRF
           DSDAASQKMEPRAPWIEQEGPEYWDQETRNMKAHSQTDRANLGTLRGYYNQSEDGSHTIQ
           IMYGCDVGPDGRFLRGYRQDAYDGKDYIALNEDLRSWTAADMAAQITKRKWEAVHAAEQR
           RVYLEGRCVDGLRRYLENGKETLQRTDPPKTHMTHHPISDHEATLRCWALGFYPAEITLT
           WQRDGEDQTQDTELVETRPAGDGTFQKWAAVVVPSGEEQRYTCHVQHEGLPKPLTLRWEL
           SSQPTIPIVGIIAGLVLLGAVITGAVVAAVMWRRKSSDRKGGSYTQAASSDSAQGSDVSL
           TACKV'''

#remove line breaks and spaces
HLA_A1 = re.sub('\s', '', HLA_A1)

#turn it into a pandas Series object
HLA_A1 = Series([x for x in HLA_A1])

#count the frequency of each unique amino acid residue
freq = HLA_A1.value_counts()/len(HLA_A1) * 100
In [9]:
#plot the frequency data
plt.figure(figsize=(10, 2))
freq.plot(kind='bar', alpha=0.8, color='orange')
plt.xticks(rotation=None)
plt.title("Amino acid frequencies: HLA-A*01", fontsize=18)
plt.xlabel("AA residues", fontsize=14)
plt.ylabel('Percentage (%)', fontsize=14)
Out[9]:
<matplotlib.text.Text at 0x81d0e87b8>

2. Save and download the notebook

After we finish the above analysis, save the notebook, and download it as an IPython Notebook (.ipynb). Don’t download it as an HTML file, unless you want to embed it as an iframe. Here this file is saved as notebook_blog.ipynb.

3. Convert and embed the notebook

Navigate to the directory where the notebook is saved. Run the nbconvert command below to convert the downloaded notebook into a simple HTML file.

jupyter nbconvert –to html –template basic notebook_blog.ipynb notebook_blog.html

The source text in the notebook_blog.html file can now be copied and paste straight into the WordPress post/page editor.

4. Update the WordPress style.css file

The following css allows you to mimic the typical look of a jupyter notebook. From the WordPress Dashboard, navigate to Appearance -> Editor then paste the following code to the end of the styel.css file.

5. That’s it, you are done!

The major downside of this approach is that this is a static version of the notebook, it cannot be re-run or shared easily. You should keep a copy of the downloaded .ipynb on GitHub for the ease of reusing the notebook and keeping track of any changes.

If you are using plugins such as Crayon Syntax Highlighter. The <pre> tag must be set to class=”crayon:false” or something similar to disable the rendering by the plugin.

Plotting the Number of HLA Alleles from IMGT Stats

The human leukocyte antigen (HLA) is a gene complex encoding the major histocompatibility complex (MHC) proteins in humans. The complex is located within the 6p21.3 region on the short arm of human chromosome 6. These genes are highly polymorphic, which means there are many different HLA alleles. More information of HLA and HLA alleles can be found on hla.alleles.org and The IMGT/HLA Database. This is an attempt to plot the rapidly increasing number of HLA alleles over time, with data directly parsed from the IMGT/HLA Statistics page. This notebook file, described below, can be downloaded here.

In [1]:
import matplotlib.pyplot as plt
import pandas as pd
from pandas import Series, DataFrame
from datetime import datetime

%matplotlib inline
plt.style.use('ggplot')
In [2]:
#read data from IMGT and put the 2nd table into a pandas dataframe
url = 'https://www.ebi.ac.uk/ipd/imgt/hla/stats.html'
df = pd.read_html(url, encoding='utf-8', skiprows=1, header=0)[1]
In [3]:
#parse the HLA dictionary release dates
df['Month'] = df['Month'].apply(lambda x: datetime.strptime(x, '%Y-%m'))
#reset dates as index
df.set_index('Month', inplace=True)
In [4]:
#plot just the number of alleles over time
#each dot represents a release
plt.figure(figsize=(12,5))
plt.title("Number of HLA Alleles Over Time")
plt.ylabel("Number of Alleles")
df['Alleles'].plot(kind='line', color='r', alpha=0.8, marker='.', linestyle='')
plt.xlabel('Year')
legend = plt.legend(loc=2, frameon=1, shadow=True)
frame = legend.get_frame()
frame.set_facecolor('white')
In [5]:
#plot with 'component entries'
plt.figure(figsize=(12,5))
plt.title("IMGT HLA Database Growth Over Time")
plt.ylabel("Number of Alleles")
df['Alleles'].plot(kind='line', color='r', alpha=0.8, marker='.', linestyle='')
df['Component Entries'].plot(kind='line', color='b', alpha=0.5, marker='.', linestyle='')
plt.xlabel('Year')
legend = plt.legend(loc=2, frameon=1, shadow=True)
frame = legend.get_frame()
frame.set_facecolor('white')
References:

Robinson J, Halliwell JA, Hayhurst JH, Flicek P, Parham P, Marsh SGE
The IPD and IMGT/HLA database: allele variant databases
Nucleic Acids Research (2015) 43:D423-431

Robinson J, Malik A, Parham P, Bodmer JG, Marsh SGE:
IMGT/HLA - a sequence database for the human major histocompatibility complex
Tissue Antigens (2000), 55:280-287

Running My Own Jupyter Notebook Server in a FreeBSD Jail

As a non-computer/data science professional, my fascination of what one can do with data science is growing day by day. I am aware that Enthought Canopy and Continuum Anaconda are both excellent pre-built  packages with intergrated Jupyter notebook and everything that I might need for doing data analysis with Python. I have been using Anaconda on individual computers for over a year and really like it. However I would love to have a central place for all my notebooks, so I don’t have to copy them around while working on my projects on different devices.

There are plenty of advises out there on the web telling people that how difficult and frustrating it is to compile and install Python and those data analysis packages from scratch (e.g. numpy, scipy, matplotlib and pandas just to name a few).  But I wanted to see how hard it really is. So after successfully set up ownCloud in my FreeNAS jail in a recent attempt, I (foolishly?) decided to have a go at setting up my own Jupyter notebook server in another FreeBSD jail instance in my home server.  This documents how I did it.

1. Setup a new FreeNAS jail
2. Install Python
3. Install Jupyter
4. Setup Jupyter server
5. numpy, pandas and matplotlib
6. Some problems and their fixes
7. Installed modules

1. Set up a new FreeNAS jail [^top]

Create Jail: Click the Jails icon from the FreeNAS admin page menu and then Add Jails. Choose Advanced Mode and give the jail a name. Uncheck the box next to VIMAGE then click OK to continue. This will take a while if a jail using the standard template has never been set up before.

Enable SSH: Launch the FreeNAS shell from the treeview menu on the left. Copy all of the files in the /etc/ssh/ to the jail.

If SSH is off by default for FreeNAS, the following line in ssh/sshd_config must be commented out before copying – it can also be edited out later obviously.

Launch a shell for the jail from the Jails tab, enable password and launch sshd service:

Or permanently enable sshd:

Close the shell, SSH into the jail then update pkg and the ports repository:

This may take a while, if there is any problem, delete /var/db/pkg/repo-*.* then redo pkg upgrade.

2. Install Python [^top]
Many of the necessary packages can easily be installed by pkg install [packagename]. Before any attempt of installing from source, try pkg first.

Some of the packages can also be installed using pip, so it’s a good idea to install pip first:


3. Install Jupyter [^top]
pyzmq the Python bindings for ZeroMQ must be installed before jupyter.

This whole installation shouldn’t take more than a few minutes. Before I figured out how to install these quickly, I went down the route of installing jupyter from BSD ports and it took a very long time!

4. Set up Jupyter server [^top]
Jupyter notebook’s online documentation details how to setup and run a secure Jupyter server. To quickly test the server is working properly, the password protection and SSL encryption can be skipped and added later when required. A default configuration can be generated using the following command:

This outputs jupyter_notebook_config.py in the ~/.Jupyter directory. There are a huge amount of options in the configuration file. To simplify things, I just created a jupyter_notebook_config.py with the following lines, instead of using the automatically generated one.

Once the above steps are complete, start the notebook server:

At this point, the server needs to be launched via SSH, it would be very annoying and inconvenient to do that. We need to add this to the crontab:

Now restart the jail and try out the brand new Jupyter Notebook server! Everything should work as expected.

5. numpy, pandas and matplotlib [^top]
Unfortunately, a Jupyter notebook server is not very useful without a few Python modules such as numpy, pandas and matplotlib.

numpy and pandas:

matplotlib:
The easiest way to install matplotlib is to follow the installation guide on its website:

The installation takes less than 10 minutes, compared to building and installing from BSD ports, this is like ‘lightening fast’.


6. Some problems and fixes [^top]
When importing numpy, pandas or maplotlib in the notebook, an error occurs with the following lines at the very bottom of the error message.

This can be fixed by setting the LD_LIBRARY_PATH to the required gcc version. This was discussed over in stackoverflow.

If Jupyter is always being called from terminals, adding the following line to /.cshrc will make the LD_LIBRARY_PATH setting permanent:

However, if Jupyter notebook is to be launched automatically after boot by crontab, this fix does not work. I have to modify the /usr/local/bin/jupyter-notebook file to change the system environment before launching the notebook server (highlighted are inserted lines of code):


7. List of all installed modules [^top]
Some of the packages below (e.g. biopython, beautifulsoup4, pypyodbc etc.) are not required but they were installed for future use, when I was stuck and got frustrated :

Conclusion
I have had spent more than a couple of days to make this whole thing working. After I was satisfied with the server, I then created another jail, and made a second attempt looking for ‘better’ ways to set this up. The above procedures take only a couple of hours in total.

To be honest, without a proper step-by-step tutorial at the beginning, for a newbie like myself, it was quite difficult! But I have learned a lot during this process, in the end I think it’s worth it.

[back to ^top]

Toggle the Input Fields of Jupyter Notebooks

Many people have described how to toggle the input <div> tags of a Jupyter(IPython) Notebooks. I believe Damian Kao was one of the first to blog about it. It’s pretty simple if you know a bit of jQuery. In fact, you can just use pure vanilla javascript too, but, at least for me, jQuery makes it easy to read; and since it is already an integral part of the Notebook, why not take advantage of it.

Here’s my version of the code:

Basically you select all the <div> tags of input class then toggle its visibility and height. I also hide the output prompts too by toggling <div> tags of output_prompt class.

You can add this code to your Notebook’s custom.js or even make it into an extension. But for me I only need it to render my exported html pages so I can print/publish them. For convenience, I upload it this script to my website, so I can point the src attribute of <script> to it from anywhere.

Before
toggle_0

After
toggle_1

Scraping Youtube Video Playlist

Youtube is not just a website for cute animal videos. I found it a great place to learn programming (and other interesting skills), especially for non-professional coders like myself. I have been trying to learn more about Python by following a number of great channels like sentdexnoobtoprofessional and CodeGeek etc. As I am progressing my “Youtubeducation”, I have developed the need to track what I have been watching along side other learning activities, I wanted to scrape the links and titles of the videos for my notes. Taking this as an opportunity to practice what I have learned, I have started coding a Python class to scrape any Youtube playlist when given a URL.

Iteration #1:

Without inspecting the HTML of a playlist webpage, I started coding. In hindsight, this was so WRONG in many ways. Anyway, you only learn by practicing and making mistakes.

On every playlist page, take this CodeGeek’s playlist for example, there are no links to other videos except those on the list. Each video is linked twice, one through its thumbnail and the other via the title text, except the first video which has 2 more links at the top as the Play All option. The link to a video looks like this:

or

So I came up with this idea of extracting all links begin with /watch?v=. Then make them into a Python set() to remove the duplicates, then I will have the list of URLs to all videos. Here is what the code looked like:

Once this rawList is generated, the indices extracted from index= in the URLs are used to sort the final list in order.

Of course there was some parsing going on using the custom function urlSplit() in the middle, but I won’t bore you with that.

This method kind of worked – input a playlist URL, it spit out all the links to its videos. But it didn’t feel right. Especially when using set() to remove duplicates, I lost the order of these videos and had to parse their indices to sort them again – this seemed counter intuitive to me.

Iteration #2:

I finally realized that I must check the HTML code to find out what’s really behind these links. As I mentioned above – I really shouldn’t have started any coding without doing so!

Inspecting HTML elements using Chrome browser is very handy. Right click the element, in this case the link to a video, select Inspect or press Ctrl + Shift + I keys to bring up the page inspection console with the entire <a> tag of the link highlighted.

Oh, Boy! Why didn’t I do it! It’s now so obvious that the video thumbnail and title links, although exactly the same, they are wrapped around <a> tags of different classes. Forget about all the duplication removal and indices, I can simply use Beautiful Soup to extract only links that belong to this pl-video-title-link class. How beautiful is that? Here comes the second iteration of my code:

Only the last line was different, but it’s simpler and more elegant. Because Beautiful Soup is parsing the links from top to bottom, so all of the URLs in the list are in the correct order, there is no need to fiddle with them. In the end I put the codes into a class for ease of usage and added a input URL validation check. Here is the full code:

As you may have spotted in the code, in addition to the URLs, I have also included the titles of the videos in the playlist – it makes sense doesn’t it? This code is packaged and can be downloaded here from github. This code is in the code folder. There is also a main.py file in the same folder showing an example of using this class.

Luminex .csv File Anonymizer v0.1a Download

Luminex csv file anonymiserSolid phase multiplex-bead arrays using the Luminex platform are widely used in many clinical as well as research laboratories. We are relatively comfortable at using our locally generated results despite the fact that inter-assay variability sometimes can be a huge problem.  However, when it comes to compare results with other laboratories, we are not so sure.  Previous studies have shown that by standardized protocols, laboratories could achieve lower inter-laboratory assay variability. This tool was created specifically for clinical laboratories who would like to collaborate, share and compare results anonymously. As its name implies, it does one job and one job only – anonymizes your .csv Luminex output file, renames the patient identifiable labeling of samples to just Sample_###.

Download Luminex Anonymizer v0.1a

This app is very simplistic and hopefully easy to use. To install the app, download it from the link above. Unzip the file into any folder on your Windows computer (Windows 7, 8, 10 32 bit/64 bit). To run this app, browser to the folder, and double click LxAnonWin.exe to launch it. Use the Open File… button to locate the .csv file that you would like to be anonymised. Click Open. If no error occurs, the anonymised file (original file name + anon.csv) will be output in the same folder as your source file.

Warnings:
LxAnonWarning
On my own computer, after I downloaded this zip file, the above warning popped up in Chrome. This may happen on your computer too. If you trust me, please ignore the warning and proceed. All the files inside are generated by cx_Freeze, a set of scripts and modules that convert Python scripts into executables. The app itself has less than 100 lines of code, if you are comfortable with compiling your own executable, or running it in your own Python environment, please get in touch for the source code. I have not tested it in OS X and Linux environments, however, this app only uses standard Python libraries, so hopefully it should work under any OS that supports Python 3.4 and above. This is intended to be a small component of a bigger open source project, however at this preliminary stage I am not ready to release all the codes at the moment.

Installing ownCloud and Lighttpd in FreeNAS (FreeBSD) Jail

I have been using ownCloud as a plugin in my FreeNAS server for about 6 months. I use it to sync the codes that I have been writing on my work desktop with the FreeNAS server at home so I can work on them wherever I go. There are a large amount of data that I am processing, using public services such as DropBox and Google Drive are not very practical and can be quite expensive. To be honest, it has served me well and I really liked it. The version I originally installed was probably version 8.0.

Unfortunately, I have this habit of hitting the upgrade link whenever the app prompted me. So far so good, until this Sunday! After upgraded to version 8.2.1, then the mapped folders on my FreeNAS server stopped showing, when I logged into my ownCloud server, even though I can see them correctly mounted when I ssh’ed into the server. When deleting and adding the jail storage for those folder again were of no avail, I ended up deleting the plugin jail and went on and install ownCloud 8.2.1 again in a brand new jail from scratch. Because I am doing it from scratch, I have done a bit of research before I started. This guide and video by Dr.KK have been extremely helpful. I followed his advice and chose the Lighttpd + sqlite of webserver/database combination. Because I am following his tutorials, the steps below are going to be same or very similar, nevertheless, I reproduce them here because some steps in the guides were not so clear to me and I’ve had hard time Googling for the answers I wanted.

1. Create a new FreeNAS jail for ownCloud
When you install ownCloud using the turnkey built-in FreeNAS plugin, it comes with Apache server and MySQL database. These are great packages but at the same time they are also resources hungry. For very few users and relatively few files, they are way overkill. On the other hand, we have very limited control over plugin jails – it can very messy trying to customizing them.

To create a FreeNAS jail, just click the Jails icon from the menu and then Add Jails. Choose Advanced Mode and give your jail a name – I use owncloud here, but obviously it can be anything you like.
addjail
Near the bottom of the advanced mode window, uncheck the box next to VIMAGE then click OKto continue. This will take a while if you have not set up a jail using the standard template before. Otherwise it finishes in only a few ten seconds.

2. Compile and install ownCloud
There are a couple of ways to log into the jail that you have just created. Either ways, if you are using Windows, you need to download putty or Bitvise ssh client. Once you are connected to your FreeNAS server via ssh, use jls command to find out the JID of your ownCloud jail then jexec [JID] /bin/csh to log in to your jail as root.
jlsYou can also enable sshd for the jail itself and ssh in directly. To do this, copy the ssh directory in etc/ to your jail in the main FreeNAS command line:

Now go back to your FreeNAS GUI, access the command line of your owncloud jail and perform the following steps:

Now log in your ownCloud jail through whatever route you prefer and:

Update the installed packages then fetch updated list of ports. This will take a while. Once it’s done:

You now have the option to disable MySQL and enable sqlite. Click OK and start the building and installing process with:

This process will take a LONG time, make yourself a coffee sit back and relax until it finishes.

3. Install and configure Lighttpd web server
Lighttpd, pronounced “lighty”, is pretty bad ass, it is very fast and lightweight. Youtube and Wikipedia used to use Lighttpd as their web server.

To install:

Run the following command to clean up the unused installation files at this point.

Configure server:
Open the Lighttpd configuration file:

Make changes to the following lines, save and exit nano:

Enable CGI support:

Uncomment out the line below “plain old CGI (mod_cgi):

In the Youtube video DrKK explained why you should choose plain old CGI over other modules.

Start your Lighttpd server to see if it works.

If everything was setup correctly, you should be able to see the log in window when point your browser to the ip address of your ownCloud server.

4. Configure ownCloud
Setup and log into your admin account following the simple steps of the initial ownCloud setup interface. These steps are relatively intuitive and straight forward. It’s not necessary to repeat them here. Once you are logged in, head to the drop down menu at the top right corner to access all the admin tasks.

Unlike previous versions, after this update (8.2.1), to enable mapping your existing Freenas drives/volumes/directories (whatever you call them), you need to install a third party app called External Storage Support. Detailed configuration steps can be found in the ownCloud Admin Manual.

5. Configuring PHP
In the Admin page of ownCloud, you may notice that your upload file size is limited to 513MB. To change this setting you need to edit the file .user.ini resides in /usr/local/www/owncloud, not the php.ini that you are told by many installation guides in the Google search results. They are probably talking about Apache/Nginx servers. For Lighttpd, this file is what you MUST modify to change the limit. Unfortunately, it appears that every time you use a pkg upgrade command to upgrade the installed packages, this file may get overwritten. You then have to change the settings again.

6. Create SSL certificate and Enable SSL support
Follow Steps 12 and 13 in DrKK’s guide create and enable SSL for your ownCloud server.
Restart your Lighttpd to test the new settings:

If everything works well:

Congratulations! Finally you have a running ownCloud that is fast and takes up very few resources.

Running Python CGI Scripts on Linux Shared Hosting

Check Python CGI Support
Most of the Linux shared hosting supports some level of python CGI scripting. To check if your hosting service allows Python CGI scripting, you can create a file and name it hello.cgi in the cgi-bin folder of your hosting account. Update hello.cgi with the following code.

If this bit of code works as shown in the screenshot below. Congratulations! Now you can code CGI scripts for your website using Python.

firstpycgi

Add Traceback Manager for Your CGI Scripts
The cgitb module provides a special exception handler for Python scripts. This module, when activated, will report back any errors and exceptions in a detailed, formatted webpage or a file if you choose to not show it on the browser.

cgitb

The report includes a traceback showing excerpts of the source code for each level, as well as the values of the arguments and local variables to currently running functions, to help you debug the problem. To enable traceback manager, add two more lines to your hello.cgi file:

It maybe pretty obvious to some people, but for me it took a while to realise that cgitb can only help you with code after the cgitb.enable() command. For example if you write your import statement like this:

The browser will show nothing that tells you abadmodule does not exist.

How to Find Out Your Current Python Interpreter Version?
To find out your Python interpreter version, modify your code as shown below:

Now you can see on the browser:
cgipyversion

If you want to know more about the environment your Python scripts are going to be in, refer to the official Python sys module documentation for details.

Useful Code Snippets to Help You Getting to Know the Environment
What other version(s) of Python is(are) available

List the sys.path environment variable

List the available external modules