Hello everyone. Needed Feedback for pip_tkinter.

Hello my fellow GSoCers. Hope you all have very good time and regularly check for your blog feeds in terri.toybox.ca/python-soc/.

I just needed feedback for my tkinter based pip GUI application in order to improve it further.  Your contribution as feedback can be very valuable and helpful to let me know what people may expect from this project. Let me tell you about this project :

We have made a preliminary version of GUI for PIP. This project is intended to provide a GUI version for “pip”  ( target audience for the project : beginners in Python or people who are not familiar with command line).

How to install pip_tkinter?

Please post as many issues and suggestions here : https://github.com/upendra-k14/pip_gui/issues

The project idea is discussed on these issues on Python Bug Tracker :
1. Issue #23551 : IDLE to provide menu link to PIPgui.

2. Issue #27051 : Create PIP GUI

The GitHub repo of the project : https://github.com/upendra-k14/pip_gui/tree/dump_code


Creating documentation with Sphinx

This week I worked on creating documetation and creating a web crawler for my new feature of providing users with option of installing packages from PythonLibs.

It’s really a great tool for creating docs. In just few steps I could create docs as compared to creating a whole website based on django or using static webpages on github.

We can create docs in few steps :

  1. mkdir docs
  2. Go to docs directory and run
  3. Navigate to

    and change :

    sys.path.insert(0, os.path.abspath('../..'))
  4. Now run
    sphinx-apidoc -f -o source/ ../mypackage/
  5. Our directory structure should be like this :
    |-- README
    |-- setup.py
    |-- myvirtualenv/
    |-- mypackage/
    |   |-- __init__.py
    |   `-- mymodule.py
    `-- docs/
        |-- MakeFile
        |-- build/
        `-- source/
  6. Finally run the following command to create html files from .rst files
    make html

We can also play with MakeFile to configure the settings based on our preference.

Tkinter with Multiprocessing

Tkinter is mainly based on single-threaded event model. The mainloop(), callbacks, event handlers and raising tkinter exceptions are all handled in single thread. That is why, it happens quite often that tkinter GUI becomes unresponsive ( for the user or is considers to be unresponsive by the user : which is a bad user experience) when event handlers try to do long blocking operations. In order to avoid it and improve user experience, it is generally advised that long running background tasks should be shifted to other threads.

But, we have two choices to incorporate parallelism in application using Python:

  • Multithreading
  • Multiprocessing

But, Python’s multithreading module is notorious due to GIL in Python. ( Google about GIL, you will find about it). It prevents Python multithreaded applications from taking full advantage of multiple processors. I used multithreading module with my application, with no significant improvement in performance. However, if we have used basic multithreading like :

new_thread = multithreading.Thread(target=some_function)

It was very easy to convert it into multiprocessing module by replacing ‘multithreading’ with ‘multiprocessing’. And also we have used the queues to send messages from processes running in secondary threads to tkinter mainloop() (primary thread), then those queues should be replaced multiprocessing safe queues : multiprocessing.Queue(). But, it is important to remember that only objects which can be pickled can be allowed to pass from process X to process Y. The general thumb rule I came to know about it was that generally all Python native objects like lists, tuples, strings, integers, dictionaries are picklable. But, complex classes or objects can’t be pickled. I found about it here.

We need to ask us questions to decided which objects can be pickled :

  • Can you capture the state of the object by reference (i.e. a function defined in __main__ versus an imported function)? [Then, yes]
  • Does a generic

    rule exists for the given object type? [Then, yes]

  • Does it depend on a Frame object (i.e. rely on the GIL and global execution stack)? Iterators are now an exception to this, by “replaying” the iterator on unpickling. [Then, no]
  • Does the object instance point to the wrong class path (i.e. due to being defined in a closure, in C-bindings, or other __init__ path manipulations)? [Then, no]

A better option is check the docs here ( a more definite answer ) :

    These objects can be pickled :

  • None, True, and False
  • integers, long integers, floating point numbers, complex numbers
  • normal and Unicode strings
  • tuples, lists, sets, and dictionaries containing only picklable objects
  • functions defined at the top level of a module
  • built-in functions defined at the top level of a module
  • classes that are defined at the top level of a module
  • instances of such classes whose __dict__ or the result of calling
    __getstate__() is picklable (see section The pickle protocol for details).

My next post will be just a code sample on how I used multiprocessing with Tkinter.

Unittest for tkinter applications

This week I really went through a lot of codebases, docs and Python books to learn about unittest. It was really a difficult task to learn it because we need to think of our application in a different way. We need to think about possible use cases of different methods, verify their functionality and logic and check for exceptions. All this becomes difficult and tricky when we need to create unittests for tkinter application.

The main difficulty in tkinter application is that we can’t just normally check it’s functionality. The call to root.mainloop() is a blocking call. Once we call root.mainloop() in setUpclass() method of unittest.TestCase object it blocks the further execution of tests which is not desired.

Ok, then as a normal person you will try to run root.mainloop() in a different thread other than the main thread in which the unittests are made. This trick also fails as neither tkinter is very handy nor unittest module when it comes to multithreading. root.mainloop() can only be run in main thread, therefore the unittests fail when we attempt to run root.mainloop() in secondary thread.

Therefore, finally we can do one thing. Actually root.mainloop() is nothing but just a loop running which periodically checks for changes in GUI elements. We can simulate the functionality of root.mainloop() ourselves by calling root.update() method whenever we change the GUI element programatically. Yeah, one more thing any GUI event can be only injected into tkinter GUI through invoke() or generate_event() methods. They work exactly similar to user input.

We can have an alternate philosophy of testing our GUI applications. It can be applied to each and every GUI application. We can test Tkinter application without using Tkinter library. Basically, remove all the non-Tkinter code from the classes
that handle the GUI and shunt that to another class. This way, we can easily test the logic that is being implemented without actually using Tkinter.

I have planned to use both methods for testing as they represent different perspectives.

Multithreading with tkinter

Recently, I got stuck with very new problem (for me) of updating GUI in Tkinter when long processes are needed to be run (like running a time-consuming loop, waiting for a process to complete and return or fetching something from URL). Actually, for processes requiring a long time to complete, the Tkinter blocks other GUI events. Because of it, updates to GUI element only happen when the process returns after completing execution.

I earlier didn’t know about the ‘thread safe’ property of python Tkinter. The main caveat here is that we can’t update GUI elements from multiple threads. Once main thread initiates the mainloop(), we can never use other thread to update the GUI. However, we can easily do background processes in other threads. But, here also we need to invoke a GUI function whenever the process in the background thread stops and returns the result. But, a GUI function can only be invoked through the thread executing mainloop().

Therefore, after reading some online resources on StackOverflow and other Python blogs, I came to know about one design pattern followed to solve this problem. Instead of invoking a GUI function, whenever the result is returned we need to maintain a shared queue. The contents of queue will be shared between the thread executing mainloop() and the thread running the background process. Whenever we need to return the result after the process ends in the background thread, we need to put the result in the queue. On the other side, the thread ( executing mainloop() ) needs to periodically check the contents of the shared queue. In my case, I couldn’t understand this concept of ‘shared queue’ just by reading about it. Therefore, let’s go through a small piece of code to understand it better.

def runloop(thread_queue=None):
    After result is produced put it in queue
    result = 0
    for i in range(10000000):
         #Do something with result

class MainApp(tk.Tk):

    def __init__(self):
        ####### Do something ######
        self.myframe = tk.Frame(self)
        self.myframe.grid(row=0, column=0, sticky='nswe')
        self.mylabel = tk.Label(self.myframe) # Element to be updated 
        self.mylabel.config(text='No message')
        self.mylabel.grid(row=0, column=0)
        self.mybutton = tk.Button(
            text='Change message',
            command=lambda: self.update_text)
        self.mybutton.grid(row=1, column=0)

    def update_text(self):
        Spawn a new thread for running long loops in background
        self.mylabel.config(text='Running loop')
        self.thread_queue = queue.Queue()
        self.new_thread = threading.Thread(
        self.after(100, self.listen_for_result)

    def listen_for_result(self):
        Check if there is something in the queue
            self.res = self.thread_queue.get(0)
            self.mylabel.config(text='Loop terminated')
        except queue.Empty:
            self.after(100, self.listen_for_result)
if __name__ == "__main__":
    root = tk.Tk()
    main_app = MainApp(root)

Here, we may need to disable the button, because it may happen that clicking mybutton may result in creating multiple new threads.

Page Screenshots in tkinter GUI

  1. Install from PyPI :welcome_page
  2. Update Page
  3. Install from Requirement Page
  4. Install from Local Archive
  5. Install from PyPI

These are some of the screenshots of the GUI application developed till now. Now, in a few days, I need to fix and give a final touch to these functionalities and write test modules for this GUI application.

Further work for next week are related to making this application multithreaded, implement advanced features and improve GUI experience. It is very necessary for me to complete the basic GUI application, so that I can get user feedback when this application is released with Python 3.6 in first week of July.