For the last class, we read chapter 2 of Joe Armstrong's thesis. Today we will be discussing chapter 4. (Note: Chapter 3 which was skipped was all about the Erlang language syntax ... I skimmed it over, but after reading Chapter 4 it is obvious that I really need to go back and read Chapter 3 in more detail)
In Chapter 4, Joe introduces us to a couple of concepts. Specifically, the chapter discusses some high level concepts for programming with Erlang such as keeping concurrent and sequential code as separate processes, the overall ideology behind error handling and the concept of letting a process die on error while another process monitors for errors (workers/supervisors).
The chapter wraps up with a brief discussion on intentional programming. I thought this discussion, albeit brief, was interesting. The general concept is that your code should be easily understood and one of the examples given was about a lookup method which could be used in three different contexts, depending on the developer's intent. Armstrong points out that a better approach would be to make this three separate methods. That way it is clear what was meant by the calling function. I think there is definitely some merit to this approach and I really don't think it is specific to Erlang.
As time permits, I would like to write a program using Erlang. I like the ideas we've seen so far and find it interesting. Any ideas for a good project to get started?
Thursday, October 22, 2009
Dense Linear Algebra, Graph Algorithms, Monte Carlo
No, contrary to how the title sounds, I am not moving to Vegas with some surefire "get-rich-quick" gambling scheme. For one of this week's CS527 lectures we have moved our discussions from structural patterns to computational patterns. The items in the title are all computational patterns for parallel programming.
Having done very little programming (with emphasis on the very little part) for large parallel systems these patterns were new to me. The patterns made sense and after reading them I think they were clearly documented. I particularly liked reading about the trade-offs and forces in play such as data movement costs vs. computation time for the dense linear algebra pattern, data structure choices for graph algorithms and threading/SIMD allocation in monte carlo simulations.
Hopefully in tonight's discussion there will be students who have work with these patterns. I look forward to hearing about their experiences.
Having done very little programming (with emphasis on the very little part) for large parallel systems these patterns were new to me. The patterns made sense and after reading them I think they were clearly documented. I particularly liked reading about the trade-offs and forces in play such as data movement costs vs. computation time for the dense linear algebra pattern, data structure choices for graph algorithms and threading/SIMD allocation in monte carlo simulations.
Hopefully in tonight's discussion there will be students who have work with these patterns. I look forward to hearing about their experiences.
Monday, October 19, 2009
Armstrong Thesis (Chapter 2)
As part of the reading list for CS527 this semester, our class has recently began reading the thesis of Joe Armstrong. The thesis is about building reliable software assuming that there will be hardware and/or software errors present.
For this week's reading we focused on Chapter 2 which introduces the architectural model being proposed. The chapter begins by introducing the original problem domain (telecom switching systems requiring essentially 24/7 operation with the ability to be updated while running). Oh wait ... did I mention that we are also going to assume that there will be hardware & software errors and that the system has to be able to work around such issues? Not an easy task!
Armstrong approaches the problem through the use of what he calls "concurrency oriented programming languages" (COPLs). COPLs typcially consist of and must support the concept of processes to serve as isolation boundaries. Communication between processes is achieved through message passing. Message delivery is guaranteed in-order between two endpoints, but is not guaranteed reliable. The other key feature is that processes must be able to detect failures of other processes and react accordingly. Armstrong introduces the Erlang language as one such implementation.
The remainder of the chapter discusses library support of Erlang for things like stable storage (storage that survives a crash) and various behaviours that assist with building these systems.
In subsequent weeks we will be reading additional chapters from Armstrong's thesis. If chapter 2 is any indication, this seems like it will be a good read.
For this week's reading we focused on Chapter 2 which introduces the architectural model being proposed. The chapter begins by introducing the original problem domain (telecom switching systems requiring essentially 24/7 operation with the ability to be updated while running). Oh wait ... did I mention that we are also going to assume that there will be hardware & software errors and that the system has to be able to work around such issues? Not an easy task!
Armstrong approaches the problem through the use of what he calls "concurrency oriented programming languages" (COPLs). COPLs typcially consist of and must support the concept of processes to serve as isolation boundaries. Communication between processes is achieved through message passing. Message delivery is guaranteed in-order between two endpoints, but is not guaranteed reliable. The other key feature is that processes must be able to detect failures of other processes and react accordingly. Armstrong introduces the Erlang language as one such implementation.
The remainder of the chapter discusses library support of Erlang for things like stable storage (storage that survives a crash) and various behaviours that assist with building these systems.
In subsequent weeks we will be reading additional chapters from Armstrong's thesis. If chapter 2 is any indication, this seems like it will be a good read.
"Event-Based" and "Map-Reduce" Pattern
Two patterns we will be discussing in class tomorrow are "event-based, implicit invocation" and "map-reduce". The event-based pattern is a little light on details, but is straightforward enough. In this pattern, there are:
- components acting as announcers (posters of events) and/or listeners (subscriber of events)
- a medium which acts as communication fabric hooking together announcers and listeners
With this pattern, one of the keys is that announcers are not aware of which listeners (if any) are present. This combined with the asynchronous nature of the system makes for a great architecture. However, I think that implementing the "medium" is a non-trivial task that is somewhat glossed over.
The other pattern, map-reduce, is another fairly simple pattern. The concept is that you have a map function that can be applied concurrently to all objects in a given set and then the results of each operation are reduced through a reduction function. This is an excellent pattern for problem domains that fit as the only thing that really has to be done is the creation of the map function and the reduction function. I would be interested to see some actual implementations of this pattern and see how well they scale as object counts increase. In naive implementations, I think there could be some potential lock contention scalability issues with the task queue and repository perhaps?
Tuesday, October 13, 2009
Refactoring for Reentrancy
One of the readings for today's class, Refactoring for Reentrancy, was on refactoring existing Java code to make the code reentrant. The benefit of a reentrant program is that the program can run concurrently on multiple cores without interference. This will hopefully allow for performance benefits when run in parallel on multiple cores.
The authors approached the problem by looking to use thread-local storage for global variables in order to isolate concurrent tasks. Other tasks such as removing potentially unsafe calls to shared libraries and handling static initializers were also required for a functional solution. In order to assist developers in making the required code changes, the authors created an Eclipse plug-in.
Overall, I think this paper follows with a trend that we are seeing where researchers are trying to bring parallel programing to the mainstream and make it more of a "norm" than an exception. While I think the ideas presented as part of this paper are interesting, in my opinion it still falls a bit short. The developed plug-in seems incomplete and a non-trivial amount of developer involvement is still required to make things work. The performance timings also seem a bit inconclusive. Are the realized performance gains truly worth the effort? I'm not sure.
Rereading the Classics (BA ch 14)
This is it ... the final chapter in the book Beautiful Architecture that we will be reading for CS527. It has been a good read. I liked the format of different authors/system designs per chapter most of which I agreed with and liked, a couple I didn't. As for this last chapter, the title is "Rereading the Classics". Classics serve an important role in education. In most cases, they help to present an unmuddled view of the topic at hand and explain/present materials in ways that seems natural and easy to understand. After all, if they didn't they probably wouldn't be classics!
The final chapter introduced Smalltalk. Having come from a predominantly C/C++/C# background, I have to say this language took me a bit to wrap my mind around. I liked what I saw. I think the language deserves a lot of credit in paving a new way of approaching things even if it is not necessarily considered "mainstream".
The last section in the chapter introduced several different structures created by famous architects. Some of these structures while considered beautiful and works of art were not necessarily the most habitable of places. One such structure was deemed inhabitable by its occupants, but yet is still looked at as a modern architectural marvel. This begs the question ... can form and function not be counteractive forces? I think so.
The final chapter introduced Smalltalk. Having come from a predominantly C/C++/C# background, I have to say this language took me a bit to wrap my mind around. I liked what I saw. I think the language deserves a lot of credit in paving a new way of approaching things even if it is not necessarily considered "mainstream".
The last section in the chapter introduced several different structures created by famous architects. Some of these structures while considered beautiful and works of art were not necessarily the most habitable of places. One such structure was deemed inhabitable by its occupants, but yet is still looked at as a modern architectural marvel. This begs the question ... can form and function not be counteractive forces? I think so.
Subscribe to:
Posts (Atom)