Monday, October 31, 2005

Brian Ray's Blog : Python: IPython as primary Shell

Brian Ray's Blog : Python: IPython as primary Shell: "IPython , a Python interactive shell developed for the scientific computing community by Fernando Perez is such a powerful tool, I just may replace my default login settings from just bash [1] directly to IPython over bash.

How may people do this? I mean really! Is it practical to cross this line?

If you are new to IPython, there is a decent intro article from ONLamp. Although, I fail to see how this article fits into the LAMP [2] world. Still, it unveils some need features like: magic: Quickly assess special IPython features. Tab Completion: auto-complete code attributes, classnames, methods, and more Introspection: Easy access to Doc Strings and internal documentation History: store a dumpable session of commands Debugger Access: internal access to pdb Run: a safe way to run a file and ignore the __name__ == '__main__', trick Macros: capture command history into a macro

All great stuff. Also mentioned in the article is System Shell Access. For example, you can use cd, pwd, and ls directly in IPython. To escape to the shell use '!' or '!!'.

The IPython Manual notes in the section IPython as system shell. I loose job control if I use iPython this way. Although, I haven't had many issues so far.

Starting IPython with 'ipython -p pysh', creates a special envirom"

I found this interesting because of recent ideas I've been drawing up for a Python Embedded Shell. The Python syntax for `foo` being taken as repr(foo) is to be removed, so I thought, "Why not have a special interpretter to expand `foo` to execute a program named `foo`?"


I went into it more in my mind, so I'll lay out the ideas here.

  • foo=10 would be taken as normal
  • print $foo would be used to print it to the terminal
  • foo would just run the program, foo (I updated from the original thoughts)
  • $foo would run a program stored in the string foo by name
  • if, while, def, class, elif, else, and for all work as they currently do.
  • That $ syntax is only needed where it would be otherwise confused for a program name.
  • `foo` would resolve to a special object representing a new process running by executing foo. This process object would have stdin, stdout, and stderr file-like object attributes. It would also have kill, pause, and continue methods.
  • New operators exist for piping. Such as foo->bar to pipe stdout of foo to stdin of bar. foo!->bar pipes stderr, instead. null is known as a built-in process object, which just pipes everthing to /dev/null or an equivalent.
Very basic, not well polished. Besides, I think I like Monad, the Microsoft Shell, which is still in Beta, a lot better than my idea or any command line I've ever used. Sorry, my Linux buddies, but a good idea is a good idea, no matter who comes up with it.

Sunday, October 30, 2005

Dynamic Linking for Module Attributes

In much Python code, foo.bar refers to some attribute of the imported module "foo". There is only ever one instance of any module, so could attributes of modules be given pointers somewhere in the module object (internally) and thus importing scripts can be compiled such that foo.bar would be resolved to some new bytecode that says "load the object pointed to at this location". The location would be null when compiled, but importing would consult a table mapping the module's attribute references in the script to the locations in the bytecode of those null pointers. This would effectively work like dynamic linking in C/C++, and would reduce a lot of attribute lookups to a pair of pointer dereferences.

Saturday, October 29, 2005

Focus, Direction, and Real Life

My wife and I have a baby on the way, my best friend moved in with us after running into some hard times, co-workers are causing my daily headaches, and my car broke down in my driveway. Where do I find time to code?

As it is I seem to barely have time for my code, and I wasn't making the progress I'd like even when I was impressed with the progress I was making. Factor in my continuing frustrations over the lack of writing and art I've been able to spend time on, and my output just looks dismal. Real Life, the good and the bad, are cutting into the things I always though real life would be made of, for me.

I pretty much know that until I can support myself and my family independantly, I don't have time for coding, writing, and art. I only have time for one. It forces me to look at it from a bussiness aspect, of which would bring me money quick enough to allow me to resurrect my other hobbies? It also instills growing fears of loss if I fail at any of them. Not only will I continue to work a dead job, but I'll continue to have no time for creativity.

Where's a grant when you need one?

Friday, October 21, 2005

Python Coroutines Schedulers

Just wanted to post a quick thought, mostly so I would remember it. I want to try to write a small decorator to turn any function into a coroutine, associated with a particular scheduler. When called, the coroutine would return a handler, and the scheduler would continue normally. it would be able to tell when a coroutine was called by another, and handle that appropriately, allowing the yields of the inner call to be yielded by the handler as an iterator. This would allow coroutines to be called pretty much just like functions, with everything being scheduled in the background pretty nicely.

Monday, October 17, 2005

PyPy and More Platforms

PyPy is coming along nicely. It gets closer to matching CPython for speed every day, but it still has its problems. It brings more questions to my mind about the matter of duplicated efforts with all of these Python implementations.

What we really need is a standardized from of representing Python code in Python code. Things like a Statement class and Expression instances, for example. This would allow people to easily write code to optimize and analyze code parsed by any front-end and compiled to any back-end.

Monday, October 10, 2005

Helping a Friend

A very good friend of mine has moved in with us this weekend. Some hard times, which were severely drawn out, lead to a four-hundred mile relocation back home, so I've cleaned out my office and set up the spare bed, to make him feel at home. Hopefully, things will get better in fast time. At least it all doesn't seem to be getting to him. Maybe, I'll get back to my old gamer-self with someone to thrash on in Halo, again.

Friday, October 07, 2005

My Take on Python Concurrency

The mailing lists and blogs have been abuzz lately with discussions, arguments, and good and bad ideas about how to deal with concurrency in Python. There is a general distaste against using vanilla threads, due to the difficulty in managing shared resources. Generators are an increasingly popular option, but they don't offer any pre-emptive nature, if you are into that sort of thing. And, of course, the Global Interpreter Lock is bashed and many pray for its removal, but that will only happen if a real concurrency mechanism is added not just as some library, but into the library itself.

Now, off the bat I'll say that I don't hate threads, but I really wish there was a better alternative. I do want to see the GIL go away. Finally, I don't like having to explicitly suspend my task every so often to allow something else to process for a while. This is tedious and does not always reflect the nature of the task well.

Active Objects seems like the most Pythonic approach, and like they could work well with existing, non-concurrent code. With Active Objects, you have objects designed to work concurrently. Calls to their methods are typically queued so that only one method of one active object will be executing at a single time. Thus, the active object does not need to syncronize on its internal state data. Extended loops in a method could get greedy, however, but that is a small matter.

The biggest issue is dealing with shared objects, which happens so much in Python's pass-everything-by-reference world. Methods can be crafted to be safer, such as by taking only basic types (integers and strings) and containers of those or containers (lists of strings and tuples of dictionaries mapping strings to booleans, etc.). But, inevitably, you will have to pass something more to a method.

One approach could be to copy objects before passing them, even to deep copy them all. Another would be to create special shallow objects that represent the state of the original without copying all of it. But, perhaps the best choice is to simply use more active objects. If we pass one active object to the method of another, and are accessing all data through their "active methods", then syncronization is generally secured. Perhaps it would also be beneficial to define a common locking API for all active objects.

Python is a wonderful, dynamic language, but there is some speculation that the very dynamic rules that make Python great, are also making it so difficult to settle on a good concurrency mechanism and to allow safe, concurrent code. Many Pythoneers scuff at the idea of true protected and private attributes, but maybe that's just what we need.

The ability to modify everything may be useful, but maybe it should be controlled a little more. We should be able to execute code in the context of protections on certain objects, like giving read-only versions of modules or write-locked lists. This becomes faulty when you realize you can just call object.__getattribute__, but I have a possible solution: the protected class. It would be a new built-in class, and anything inheriting it would be given special abilities: object's methods would treat is specially and following its access rules (properties would be immutable, exception through the functions defined for them, for example) even when using object methods directly, or other introspection mechanisms. Combining this with crafted globals and built-ins for 'jailed' code, could be a sucessful way to execute multiple lines of code in parellel without worrying they will step on each others' toes.

Sunday, October 02, 2005

Python Object Space Demo

I've whipped up a little test of object spaces. You can create global contexts and run code within that context. You can also queue functions to be called (the API forbids multiple threads per space), and set response functions to be called at the return of a function called in context of the space. It is very simple, but effective, I like to think. The initial code is available here.

Saturday, October 01, 2005

Branching out with Blogging

I may be creating some new blogs. I hold back posting some things, because I want to post on so many topics. I'm considering keeping this blog for general ranting, political commentary, and the like. I may also create the following blogs:
  • Ranting and Raving about Games and Movies
  • Ranting and Raving about Software
  • Ranting and Raving about Art
The last one is likely not to be created, although I may resume my art and end my very long hiatus from Deviant Art (see my DA page here). This will allow me to more easily and openly blog about more specific things more often, without worrying that no one will read a blog so mixed with topics so unrelated.