Wednesday, September 05, 2007

Caught in the Fire

Stop the presses.

Time for you to update your links. I will from now on be blogging on the following location:

mythology.nu (link)

And that will be the name of the blog: Mythology.

For some reason I can't export my posts from this blog to that one, so I'll keep this blog as an archive for the time being (perhaps forever).

I will conclude this blog by explaining its title, Caught in the Fire.

Firstly, I started this blog as a kind of self-therapy, dealing with some personal problems. Caught in the Fire seemed like a good title (and it is). While I did write some insightful posts on that topic, it was difficult and I never dared announce the blog to anyone (except a few friends). So I removed those posts and began writing about other things that interested me. With great success -- it's actually both more therapeutic and more fun this way, and I got some great readers and commenters that I wouldn't have had if I had kept the original topic going.

Secondly, it's a line from a song called "Walk through the Fire", which is a song in an episode of Buffy (Once More with Feeling -- a musical, and possibly the best ever TV show episode).

Exit fire, enter mythology!

Tuesday, August 28, 2007

anti-aging: overpopulation

I wrote some posts a long time ago regarding the possibility of being able to stop the aging process [edit: actually, only two posts it would seem. I had planned more...] some time in the future (which seems like a likely possibility), specifically arguing against the arguments against non-aging. A substantial number of people dislike (even hate) the idea of non-aging, and prefer to age and die. I don't.

One objection that I didn't say anything about is that of overpopulation. If we don't age, the rate of dying would naturally decline (but it would not reach zero - people would still die from other things: accidents, sickness, murder...), and the population might start growing rapidly.

If so, is that a problem?

The growth might not be as dramatic as one might think, for several reasons.

Firstly: the age in which people have children now is adapted to our life-span. People can't at present have children at age 200, because they won't be alive. But that will change completely if aging is stopped. There will be no age-related restriction and people will be free to get children whenever they want. And I suspect many will wait. Humans are often pushed into things they don't really want to do, because people expect them to do certain things or because they can't wait because it will be too late. Those expectations will disappear as the age in which it will be "too late" will disappear, because it will never (theoretically) be "too late". And if people get children at age 200 rather than 25, that will make a difference in population growth.

Secondly, much of the population growth today is in primitive and poor nations (often influenced by religious ideas geared toward child production in various ways), like parts of Africa and the Middle East. In the West (including semi-West nations like Japan), on the other hand, the population is actually shrinking (and would be so even more if the immigration groups, still influenced by their earlier culture, hadn't been included in the statistics), which is seen as a big problem by many. In case of non-aging this unwillingness to reproduce wouldn't be a problem anymore, but an asset.

Thirdly, if overpopulation is starting to become a problem, it will get rather unpopular to get children. It will be seen as irresponsible and bad. That certainly will affect peoples desire to reproduce.

Fourthly, if overpopulation really becomes troublesome we will, as a last resort, regulate against it, and have some sort of system with a queue, and let one child be born for each person that dies (or something like that).

Fifthly, at some point in the future we will probably (assuming a continued development of science and technology) start going into space big time, and start colonizing other planets.

Finally I got to post this one. I've had it in my draft for some time. I've lost most of my interest in the topic, and I have more pressing things to attend to. If anti-aging technology is developed and put in use before I die, fine. If not, then I get to know if something is waiting beyond the rim. Fine too, unless it's a hell of some sort. That would really suck.

Friday, August 24, 2007

The PGA and PGU of chess tactics

Finally. Back in March I wrote about the CCT-rule. It was my intention to continue writing on that, and I did write a draft but it's not until now that I finished it. It could be a good idea to start reading that post, but I will try to make this one self-contained (which means some repetition). Also, the delay means I can write this one using the terminology of PGA and PGU, and using a diagram position that wasn't available at the time.

I still think that the CCT-rule (checks, captures, threats) is the by far best method to find tactics, especially simple tactics (which is especially important in blitzes, but obviously everywhere.) And it is a pure PGA, as against a PGU. (Learn about PGA and PGU here. Here's the very short version: PGA stands for principles of action, and PGA is principles of understanding.)

There are many PGU's in chess, including categorising different "motifs" like forks and skewers. But I've found it to be useless to actually systematically try to go through these in a game. It's a very useful skill to see these motifs quickly, and these terms are very useful in some ways, but non-useful to consciously go through one by one in a game. It's just too much, and too incomplete. The CCT-rule on the other hand is complete. It does cover all tactics (which doesn't mean that it guarantees you will find any tactics, of course, just that there is no tactic that doesn't begin with any of those.)

Checks, captures and threats and mentioned from time to time in many places, but it's mentioned together with lots of other stuff, some of it rather unimportant. (That's how I found it, literally as one of 100 or so ideas throws together on a web page). That indicates that its proper role and value isn't completely discovered.

I've found that there are several benefits to this scheme.

1. Covers all tactics (as mentioned above), which gives a feeling of "completeness", and sense of control (based on a genuine increase in control, so it's not an illusion.)

2. It gives a sharp line between categories of moves, making it simpler to mentally keep track of what you are doing. If you try out moves randomly, or based on some feeling about "importance", you basically have to keep track of the moves one by one, which often, for me at least, leads to chaos. I miss checking some moves, I double-check others, I jump back and forth between moves in a wildly irrational way. With the CCT-rule and that sharp line I get some sense oif what I've tried. (I still jump back and forth some, but not as much. There is a noticeable increase in control)

3. I often find tactics hidden behind a non-intuitive move that I otherwise may not have found. For example, you will easily find all queen sacs if they are not too deep. The problem with much tactics is that you don't even consider some moves (such as a queen sac, unless obvious), but with the CCT you do. There is no guarantee you will see the tactic just because you examine the first move, but you increase your chances anyway.

4. If you apply the CCT-rule consistently when doing tactics, the mind eventually starts to look for those moves automatically (as against in the beginning when you constantly have to remind yourself to go for these first, which feels kind of unnatural if you're used to the chaos-method of checking moves randomly based on 'feeling'). In other words, you look at the position and without any effort at all pick out the most crucial moves. That's a great thing. (No I'm not there myself completely, for the simple reason that I don't practice much, but I've seen some movement in that direction.) Doing the CCT will eventually become with feels right and natural.

For example: Take a look at the position below, it's from this post by BDK. He says: "One problem with trying to classify everything is that I become somewhat blind to tactics that don't fit into my schema.For example, here is a problem that I ran into today at CTB, with white to move." (Referring to the diagram below.)

That's one problem with using a schema of motifs. However, using the CCT-rule, the correct move, Bg8+ is literally among the first two moves to look at (which in fact is what I did look at immediately when I saw that diagram, and I solved it within a second, and that's not because I'm a great player, which I'm not), since it is one of two checks (and checks come first.) And once you examine that move, the rest comes naturally. Here's the diagram:



There are some difficulties (areas that need further development) with CCT.

Firstly, the T part. Because it isn't obvious what is a threat on the board, as against captures and checks. I've done some thinking and experimenting, and it seems useful to have the following two subcategories to find threats:

1. Look for unguarded pieces and see what moves you can make to attack that piece (often making a double-threat). (Unguarded pieces is something most player know to look for, but it's treated more or less as a random idea, just as the CCT-rule itself. Here, however, it's sorted under the T-part in CCT. It's part of a systematic, and realistic, approach.)

2. Look for moves where pieces of lesser value attack more valuable pieces (such as attacking a trapped piece by doing a pawn move).

Secondly, what to do after the first move. I admit that I mostly only apply the CCT-rule when doing the first move. However, it makes sense to apply it (in some way) further within the search trees as well, at least the second move. But it's tricky and maybe some simplified version is needed. I don't know, my plan is to automatize the CCT-rule as the root to begin with, and then worry about the rest. But eventually that question will become relevant, I think.

Wednesday, August 22, 2007

It all comes down to purpose

I wrote in this post about how purpose is central in both GTD, chess and Objectivism. And I mentioned briefly in this post the centrality of purpose in fiction writing (both on the part of the characters and the writer).

(And of course, "knowing what you want" is almost like a cliché in our culture -- even if most got the meaning of it wrong in one way or the other --, and it's a part of countless systems. Aleister Crowley spoke of a "True Will" [which basically is a mystic version of Ayn Rands "central purpose"], to be arrived at by means such as rituals and getting in contact with spirits.)

The GTD idea of altitudes (ranging from 0 to 50000 ft. ), with 0 being the purpose of the immediate moment and 50k ft. the purpose of your life on earth, corresponds nicely to the idea of different levels of purposes in stories, as I wrote about in last post. On a scene-by-scene level the most obvious purpose is the short-range zero feet purpose, but from a higher altitude, looking at many scenes at the same time, the purpose behind the narrower purposes becomes visible. And all adds up to something (assuming it's not a character who lacks a purpose, but then that usually means the character is looking for a purpose -- which is a kind of purpose --, and all characters, purpose or not, have a motivation of some kind.)

Well, I just find all that fascinating, the way in which different subjects tie into each other.

Tuesday, August 21, 2007

Storytelling: mindmap, change of method

Sorry about the delay.

I dumped the idea of writing a review of every episode on Babylon 5. The idea was to learn about storytelling by means of analysis. However, it's just too time consuming doing it that way, and I think I'll learn more from just watching the episodes and trying (in my mind) to analyse as I watch them (and pause when necessary). I need quantity to build up a large internal base of knowledge from which to draw later inductions.

I'm working on a mindmap to aid my analysis. It's a map containing info like the concepts I gave in the "concepts, method" post below. It's designed to be practical, so I ruthlessly cut out every part in it that isn't useful (which I usually find out after having watched some episodes trying to use it...)

My latest addition is a node called "scene" with some sub-nods about things related to scenes, for example what the character is trying to achieve in it. I already was on the lookout for the general purpose of the character, but there is usually some more specific purpose within the individual scene. For example, the main purpose of a character might be to climb a mountain, whereas the purpose in specific scene at the bank might be to get the banker to grant a loan so he can afford the climb.

And as soon as you have the purpose identified, it becomes easy to identify many related things, like obstacles and conflicts.

Of course, everyone notices these things anyway (or they wouldn't understand the story), however there is a world a difference between explicitly identifying these things and just getting them as an ordinary consumer. Not in terms of understanding the story or getting more out of the experience, but certainly in terms of understanding the art of storytelling.

And the more you train yourself to notify these things automatically, there deeper it's possible to go. I often get frustrated when engaging in this activity, because I sense that there is so much more to notice in the story, except that I don't notice it. It's there, I react to it as a consumer, but I can't identify it explicitly. Like being outside a mine of gold lacking the tools to access it. Very annoying. But I'll get there eventually.

Meanwhile I'll keep working on that mindmap

Tuesday, August 14, 2007

Storytelling: plots

One interesting PGU in writing is to list the number of possible (basic) plots. Some (many?) writers have done some thinking on that, and come up numbers ranging from 1 ("stuff happens") to 100, and anything in between. One popular number seems to be 7, as in this book called "The Seven Basic Plots". Here's one writer claiming that there are 20 plots, and here's 36.

The usual hierarchy applies: how many plots depends on what level of abstraction. The more detailed, the more plots, obviously. With enough details there are as many plots as there are stories. And with everything abstracted away from you get one single plot: "stuff happens", which covers pretty much all stories (but isn't very helpful in any way).

I think 7 plots sound about right; the right level of abstraction for most purposes. I will make it one of my (ongoing) assignments to identify which those are (maybe I'll agree with one of those who already have decided upon 7 specific plots. Maybe not.)

Monday, August 13, 2007

Fiction Writing: concepts, method

As with most other subjects (including chess), fiction writing is one of chaos. The books and guides lump together fundamental concepts and ideas with trivial and unimportant ones (without clearly specifying which is which), and it's up to the aspiring writer to make sense of it all and put it into a system. (Actually I'm exaggerating a little, the guides differ in quality a lot -- I'll get back to that later.)

My main task for the moment is to identify the central concepts of storytelling and then make them come to life by tying them to those concretes that they are an abstraction of. And that last part I will do by making identifications as I watch TV shows. After all, that's what being a fiction writer is all about: beer, peanuts and TV.

Right now some of the stuff I keep explicit track of while watching is:

arcs
plot
conflicts, obstacles (inner, outer)
characters purposes (what they want)
resolution of conflicts
set-pieces (often a confrontation of some kind)

I often also try to explain to myself the purpose of various scenes, and ask myself things like what would happen if it was removed (a well written piece shouldn't have any scenes that can be removed without suffering in some way.)

Sunday, August 12, 2007

Fiction Writing

TV shows and movies have awaken in me an interest in storytelling, so I've decided to add a new project to my list: learn how to create stories and write them down. Fiction writing. That's something I on and off have considered before, and I've even bought a number of books on the subject. But the time hasn't been right until now. If it is, I'll see about that.

This means a new system to play around: the identification of central concepts (and their interrelation), and the relation between concept and implementation, and so on. Very interesting. I guess there is a risk I never get around to actually write, I guess I just have to wait and see.

There seems to be interesting similarities between chess and storytelling, at least on an abstract level (things like the relationship between abstractions and concretes, and so on. Both subjects are similar in that they combine theory and practice more evenly than many other subjects. Or so I think, but I'll have to think some more about that and get back to it.)

Tuesday, August 07, 2007

Babylon : Report

A new post on my chess program. I've mostly been improving the design, but some improvement in playing strength too.

New games included in the end.

Most of what I've done since my last report is structuring the code. The major change is large scale modularization (my fifth design principle, identified here .) Earlier I had all code in the same unit (which also means same file), as it's called in Delphi. Now it's divided into 6-7 units (one main unit, one unit with classes related to evaluation of a position, and so on), and the best part is that no two units are dependent on each other. It's always one way only. This meant I had to spend considerable time decoupling different parts of the program; unnecessary connections I've made out of laziness. (That's often the case in programming -- ugly and complex solutions are often easier and faster to make, until the program starts to breakdown out of sheer complexity, in which case finding bugs and making modifications start to take much time. On the other hand, simplicity in structure and design is more difficult to achieve, but much easier to maintain, understand and modify.) Took considerable time, but it could've been far worse. I feel I understand the code better now.

Decent before, better now. So on a global level I'm pretty happy with the design now. On the level of algorithms it's still pretty messy (you wouldn't believe some of the solutions I've created and still use). But that's within controlled areas so to speak, so it's not urgent to fix.

The improvement in strength consists of two things: improvement in evaluation, and more importantly: iterative deepening. It is a complete iterative deepening with extraction and insertion of a principal variation (these terms are explained below). However, it doesn't work as it should. The PV thing is actually making it somewhat slower that before (the opposite should be the case). But, the improved time management inherent in iterative deepening is a big improvement, especially in the end game.

What iterative deepening is: playing without it (usually) means searching to a fixed depth. For example, the chess engine always thinks 5 plies deep (+ quiescence search, if included), regardless of position. But that makes no sense, because in some positions 5 plies may take 400 seconds, and in others only 4 seconds. Iterative deepening solves this problem: it means to start with 1 ply deep, and then 2, then 3, and so on, until the time runs out, and then choose the best move from whatever depth you managed to achieve given that amount of time. (In the games below Babylons depth varied from 1-9. Without iterative deepening I would have had to go no deeper than 3 plies, to be sure it doesn't get stuck in some time consuming analysis. And going 9 instead of 3 is a huge benefit.).

That thing "extraction and insertion of a principal variation" means the following: when the engine goes, say, 3 plies deep (expecting to go more), it extracts what is considered at that point to be the best variation (both sides playing the best possible moves). And then at the next iteration (4 plies deep), the variation extracted the iteration before is the first variation to be tried. This improves the move order, which improves speed. In fact, it's the (by far) best move ordering principle that chess engine programmers have come up with so far.

Time for the games, three of them.

Against Raspoeting (one level in the chess program called Nagaskaki, which is free for for download . It's a very nice chess program.) A draw!




Another Game. Same enemy as once before, The Thinking Machine. Some funny moves, including a gambit (?) of some sort by TTM early in the game. It's a draw, but only because Babylon, who is two queens ahead in the game, can't differentiate between mate and stalemate.




Below a win against another old enemy, Homeostatic Computer Chess Player. It's very slow, so I stopped when Babylon had rook + three pawns against a lone king. (Technically it could have ended in a stalemate for the reason mentioned above. I'll fix that some day, just a detail.)

It's a long and somewhat boring game. A fierce warrior Babylon isn't. It's the most cautious engine around (and always ready to accept a draw, judging from all attempted threefold repetitions. But the enemy wouldn't have any of that, luckily.)

Sunday, August 05, 2007

Grand Master Chess for free

Just thought I'd tell you that Grand Master Chess (14.95$) is given away for free today at give-away-of-the-day. (I've been checking GAotD every day since it started, and found some really valuable and useful software.)

I'm downloading it right now, so I haven't tried it yet.

Update: Tried it briefly now. Disappointed. It's flashy, but no real configurability (except for graphics and a few other things), and you get no info on search depth and stuff like that (which is of interest to me). At least not as far as I have found. Also, it's kind of annoying to use. Lots of different boards and pieces to choose from, many of which I doubt anyone ever uses, so it's just pure bloatism. It has network capability (and can act as a server and client), which is good. But I suspect that it only works its own protocol (so it can only connect to itself, and not to FICS for example). Anyway, I'll keep it for now and maybe explore it some more later.

Update 2: Maybe I was too harsh. It does look great, which I appreciate.

Still, the old free version of Fritz (see link in left frame) is better.

Saturday, August 04, 2007

two sets of principles: PGA and PGU

Some time ago a wrote a post identifying positional elements. A couple of commenters seemed to assume I was trying to create a guide with elements to go through as part of the process of evaluating the position. That, as I explained, wasn't really my intention. I just find it a helpful way to think about chess (and other subjects) when not playing, which in one way or the other will affect playing as well. Hopefully.

I think it's a good idea to differentiate between two different set of principles (not just in chess but generally): principles to guide action (let's call it PGA), and principles to guide understanding (PGU). The latter is usually in the form of a classification.

For example, the period system (which I did mention in that post mentioned above) is clearly an example of PGU. No one would during a chemistry experiment have as a guide to "go through all entries in the periodic system, one by one, and ask the following question..." or something like that. That would be a complete waste of time. Yet the period system is obviously a very valuable classification. But a PGA needs to have much fewer principles and be easier to apply.

I do think that PGU can be used as a sort of knowledge base to refer to when creating PGA (to make sure that it covers enough ground, etc), as well as a knowledge base to (intuitively) draw information from when applying PGA.

The list of design principles of programming (posted recently) was a PGU, but I still from time to time, in a non-systematic way, use it to influence the process of programming. But mostly I just compiled it to strenghen my understanding.

Thursday, August 02, 2007

Battlestar Galactica -- 2003 Miniseries

I'm planning to go through all episodes of Battlestar Galactica and review them as I go along. I'm not used to writing reviews, but hopefully I will learn in time.

Actually no, I've decided to do that with Babylon 5 instead, but I had already written the first review of BSG when I changed my mind, so I'll post that one.

The BSG Miniseries (the pilot):

Obviously, spoilers abound.

Background: Humans create cylons (intelligent robots). They rebel. A war follows, which ends in a draw. The cylons leave into space. Then no one hears from them in 40 years, then they come back with a vengeance. They destroy almost all of mankind; everyone except a fleet containing about 50k humans. The biggest ship in that fleet, a battleship, is called Battlestar Galactica.

The starting point of the show is the same day they come back, a few hours earlier. Everything that happened before that we learn through exposition.

Before the attack we are introduced to a number of characters. The setting in which they are introduced is mostly 'daily life' situations. That's probably deliberate: they want show a contrast to the rather extreme premise of the show (most of mankind exterminated). One moment they drink coffee, cut their hair, and drive their children to school. The next a cylon fleet kills them all. So long suckers. They don't actually do exactly those things I said (maybe the coffee, I don't remember); those are just metaphors for everyday life. Also, they don't kill them all, just almost.

The everyday life phase is not entirely everyday though; it's a heavily slanted towards explaining that whole cylon thing, through various devices (such as a speech held by a general -- "it's now forty years since the cylon war, yada yada", and a guided tour of Battlestar Galactica, which the viewers happens to catch some of). Which of course is a good thing. The viewer needs to know those things, and it's not too intrusive (especially since it's three hours long. With a two hour movie I'm sure much of that would have felt artificial and rushed, but now it's pretty well done.)

Then the cylons attack, and they barely manage to escape. The attack is a well planned one and they aimed for total destruction, but there are specific and believable reasons why some humans survived. I won't go into that.

The worst part of BSG in my view is the storytelling. The basic design (few surviving humans trying to flee the cylons and finding a new home) is interesting, as well as many of the ideas used (the sense of paranoia created to the fact that cylons look like humans, and can even be programmed to think they ARE humans), but the implementation, the actual plots they build with these ideas, isn't top notch. [Actually, it may not even be the plots that are the problems, but their implementation. The series of events, described abstractly, could be good. I'm not sure. I know I dislike the dialog; that much is for sure.] I cannot at this point satisfactory explain exactly what's wrong (except for a few things), but hopefully it will be clearer in time. The flaws in this regard are especially evident when contrasted against the excellent storytelling in shows such as Babylon 5 and Buffy.

And I feel the characters aren't believable at times. Here is one example: The (newly appointed) president, who has cancer, says at one point (after the extent of the cylons ongoing attack is known, and her surviving this attack is uncertain):

"I wish I could say [the cancer] was the least of my worries. But the world is coming to an end, and all I can think about is that I have cancer and I'm probably going to die."

Come on! You are on the brink of being killed by cylons (a much more acute threat than the cancer); almost all friends and family, as well as most of humanity, has been killed already, and you worry about the cancer? Not to mention that fact that the cylon attack is still very new, and it should be a rather shocking thing to experience. Her reaction is completely unbelievable. And spare me any explanation such as "yeah but humans aren't always reacting rationally you know; blah blah". That's nonsense, because my objection isn't that her reaction isn't "rational".

Sometimes the dialog is slightly better, as in the following exchange: (It's before that attack. A cylon is speaking with the scientist who unwittingly, but still immorally, helped the cylon attack. He has just found out about what he's done (but doesn't know the imminent attack), and is worried that he might be charged with treason and executed.

Cylon woman: "It won't matter, because in a few hours no one will be left to charge you with anything."
Gaius: "What exactly are you saying?"
Cylon woman: "Humanities children are returning home. Today."

The look and sound of BSG is very good, but I won't go into that.

Another good thing about BSG is the drama and tension created through having to consider (and make) tough decisions, which includes sacrificing the innocent. (However, 24 is probably even better in that regard.) I'm tired of last-second-rescues and TV shows where nothing ever really is at stake. At one point the commander in chief has to decide to leave a fleet of civilian ships (including a little girl in an artifical garden she had talked to earlier) to certain death because of approaching cylon attackers.

Unresolved issues:

Escaping the cylon fleet (and finding a new home -- they actually look for earth; the show starts in another solar system, and earth is basically just a myth to them, but they need something to search for, purpose of some kind, so that's earth.)

We get to know that one member of the BSG crew (and which one) is a cylon without knowing it. That's a good a plot device which is used frequently is various ways in later episodes (not just, or even mostly, in regard to this particular crew member.)

"Father and son conflict" between two of the main characters.

Immoral scientist, believed to be good by the others, is on board the ship (in close cooperation with the leaders).

The mentioned scientist is haunted by an unexplained woman only he can see.

Conflict between Starbuck, a main character, and a superior officer.

That superior officer has a drinking problem, which is bound to cause problems later on.

Sunday, July 29, 2007

MBTI, more on

One problem with the borderline cases is that the difference between two types is bigger than one letter might suggest. For example, just judging by the letters J/P it's easy to conclude that the difference between, say, INTJ and INTP is minor. After all, three letters are equal. However, that's not the case.

To really understand MBTI, one has to go beyond the individual letters and see what they mean in terms of functional analysis. For example, here's a comparison between INTJ and INTP:

INTJ:

Dominant function: Introverted Intuition (Ni)
Auxiliary function : Extraverted Thinking (Te)

INTP:

Dominant function: Introverted Thinking (Ti)
Auxiliary function : Extraverted iNtuition (Ne)

(There are also a third and a fourth function, but I leave those, except to say that they, like the first two, are different in INTJ and INTP.)

So, being on the fence between J and P in this case means being on the fence between two completely different personalities, and not between two personalities that are 3/4 the same.

Some types are more similar than others. For example, INFJ and INTJ, as I mentioned, share the dominant function Introverted Intuition, so it makes more sense to be on the fence between these two than between INTJ and INTP (there are always exactly two types that share the same dominant function).

Though I might add that I don't think that MBTI is exact enough to really withstand this kind of analysis. It becomes speculation with references to theories rather than correspondence to reality.

Thursday, July 26, 2007

MBTI

Some ten or so years ago I had some interest in MBTI, a psychological system that divides humans into 16 different personality types.

I think there are some serious problems with this system, but there is some truth to it and it can be fun to play around with.

I first tested out as INTP, but not long thereafter I discovered that I'm probably more of an INTJ. Recently I've become increasingly convinced that INFJ might be the best fit. They are not that different, INTJ and INFJ, as they share the same dominant function (introverted intution), and I find myself somewhere in the middle.

Both are system builders, but INTJ:s tend to be more concerned with technical stuff whereas INFJ:s are more people oriented. (That's not the only difference, and it's not a defining characteristic.)

Anyway, I think of the nerd within me as the INTJ (whose interest are things like programming and chess), and the activist within me as the INFJ. (You don't see much of that latter side in this blog.)

INFJ:

They put a lot of energy into identifying the best system for getting things done, and constantly define and re-define the priorities in their lives.

: )

Wednesday, July 18, 2007

Basic design principles of programming

I'm going to try to compile a list of the most fundamental design principles of programming. The current list will most likely not be the last one, but I have to start somewhere. I want perhaps 4-5 principles (although it's not entirely up to me to decide -- it depends on the nature of the subject).

The criteria of a fundamental principles, which probably will become updated in time, is something like:

1. The principle should be broad, cover a lot of ground. So a design pattern doesn't count, it's too narrow. On the other hand, the principles these patterns are based on (using interfaces, loose coupling, etc) are candidates.

2. Not be trivial. The principle could (and maybe should) be easy to understand definition-wise, but not be easy to apply fully. (A counterpart in chess would be a positional principle like piece activity. Takes 15 seconds to understand the meaning of, but a lifetime to apply fully. Both a beginner and a grandmaster can study aspects thereof.)

3. I should have experience with it directly. There are some design principles that sounds great and useful that I've read about, but that I haven't thought in terms of and haven't seen in action directly. They may be great, I just don't know yet.

Also, it should, of course, really be a design principle, and not a design goal. You can't say for example "my design principle is to write efficient and correct code. Yihaa!", because that's a goal. It doesn't really guide or tell how to achieve that, just what to achieve. And I don't know what "yihaa" was supposed to contribute. Be serious, and cut that silly stuff out, okay? Okay.

What I've come up with, so far: (There are no revolutions here; these are well established principles, but they are usually not compiled into a list like this, or they may be mentioned together with less fundamental principles etc. The subject is somewhat chaotic, perhaps like the pre-modern area in chess was. So I'm basically extracting and compiling my personal collection, but I don't originate anything really new. Maybe someday I'll become an evil nerd-genius, but until then I'm just a lost farm boy.)

  1. Avoiding duplicity. This applies on many levels, anything from using a loop and a table instead of 30 assignments in a row, to merging two big units of some sort. Usually what is merged isn't literally duplicates, but it's similar enough so that you can extract one aspect of it and put it separately. Or something like that.

  2. High Cohesion. I think of this as 'classification'. It's basically putting related instructions (and data) in the same place, neither having non-related info in the same place nor having highly related info spread out. So it's clearly related to avoiding duplicity (but not the same), and this also applies on all levels: functions, classes, units, libraries, whatever. I once read an interview with one the programmers behind Age of Empires, and when asked for some programming advice, the only one he gave,as I recall, was having a well defined task (no more and no less) for each function. Well, the principle of high cohesion covers that, and more.

  3. Loose coupling. Often mentioned together with high cohesion. It means not having unnecessary connections between different parts of the program. A simple example: Three parts, A, B and C. A calls both B and C, and B calls C upon receiving a call from A. Now, instead of B calling C it's, or could be, possible to make B return whatever value it sends to C, to A and then let A send it to C. That would eliminated the (direct) connection between B and C, with A as an operator. Assuming that's a reasonable task for A (which it probably is since it already has contact with both anyway.)

  4. Encapsulation. As in information hiding. Maybe this is too object oriented to be included. But I feel three principles are too few, so I'll include it for now, and maybe remove it when I have 1-2 new principles. Or maybe I keep it, because OO is growing stronger by each day, and is, in my opinion, the best paradigm. (Let the flame war begin...)

  5. Modularization. Another principle with close ties to the other principles, yet different. It's breaking up the code into different units, each as independent of the others as possible. Loose coupling, mentioned above, is an important principle in achieving this.


That's it for now.

Thursday, July 12, 2007

TV shows, top ten list updated

This is an update from the list I published 5:th of May.

It's almost the same list, but with some extra fluff like links and pictures. And it's a top ten now, from being a top eight, since I've finally found two more shows worthy of inclusion (Babylon 5 and 24). I'm probably the latest kid on the block to discover 24, but I'm usually years behind in that sort of thing.

As before some shows share the same spot, but it's otherwise in order:

  1. Buffy the Vampire Slayer
  2. Angel and Babylon 5
  3. House M.D, Firefly
  4. 24
  5. Veronica Mars
  6. Battlestar Galactica
  7. Star Trek: the Next Generation, Lost


Monday, July 09, 2007

Babylon Vs Fritz, SEE now implemented, threefold repetition, more on q search

Some results: Babylon easily beats Fritz set to a rating of 1350 (lowest possible). It became a threefold repetition at rating 1500 (with big advantage for Babylon), and another threefold repetition when Fritz was set to 1596 (with equal position, endgame). I claimed some time ago that one goal was to beat Fritz set to 1350, so that is now achieved. Of course, I don't know what this rating means in relation to other scales (other than that it's much inflated compared to elo), but it ranges from 1350 to 2150. (It's btw an old free version of Fritz, and the personality used was Reckless.)

I will soon have to something about those threefold repetitions. It's a well known problem with chess computers and something all engines need to deal with one way or the other. Most implement hash tables (to remember positions) sooner or later. I probably will too, but not anytime soon. Instead I will do the following: Always save the two best moves (instead of one), and never play the same move that was played two moves ago (assuming the position is good enough, and that the second best move is good enough). That will not stop all threefold repetitions, but at least the most common one, like 14.Rba1 Rab8 15. Rab1 Rba8 16. ... it will break the potentially repetitions pattern here by not playing Rba1). Also, by saving two moves I will be able to create some randomness, by giving it a certain chance of playing the second best move (if it's good enough). However, for testing purposes it is better having it playing deterministically, that way I can more easily spot errors (by detecting deviation from previous play when it should play the same).

I've discovered that having a quiescence search not only helps in the middle of tactical sequences, but also plays an important role in making the engine play positional. The reason is that without it, the computer will always think that it can take a piece at the end of the search (or that the opponent can do so, depending on whether it's an even or odd search depth), near the horizon beyond which it cannot see. And it will adapt its move choice to that and play more non-positional, even in a completely quiet position with no immediate captures possible. Well, I just find that interesting, but not so relevant now that I have a fully functional quiescence search.

And I've also implemented SEE to improve the move order (and therefore search speed) for the q search. The result is impressive: one position that earlier took over 600 seconds to analyze now takes only 21s. Usually the difference isn't that dramatic, but it's always faster. However, I still can't go deeper than 3 (complete) plies within reasonable time, so I haven't gained anything strength-wise by using SEE, but it's faster with three plies and also pretty close to being playable at 4 plies. I consider anything lesser than 10 seconds per move in average to be "within reasonable time" (then it can play 2 12:s, with some margin). (Btw, 3 and 4 plies may not sound like much, but with q search it can still play pretty decent chess. The q search often makes it go deeper than 20 plies into the position when analyzing captures.)

Next stop is iterative deepening (again, but this time I'll make it right and not give up easily.) That'll speed up things a lot I think, maybe even get me 5 plies deep.

Friday, July 06, 2007

Found opponent. Two new games. And who's Babylon?

I've found a chess engine under current development that is of similar strength as mine (two games, both draw, will follow below), and that has been developed during the same period. That's pretty cool, as I need competitors of similar strength. And maybe they will progress together.

Meet Vicki. There is also a related blog, but it is not updated frequently.

Vicki has some fancy stuff not yet implemented in my program, like SEE (a function to improve the move order of captures, which prunes the search tree, which improves search speed) and principal variation with iterative deepening (also a way to prune the tree, but not just for captures). I did try to implement this last thing, but it went wrong and I moved on to something else. But I'll be back on that. But first I'm going to implement my own SEE-function, since it is of great help to the quiescence function (which is a real bottleneck right now).

I downloaded Arena yesterday to be able to play with chess engines. But since my program isn't connectible to it yet, I have to be the "operator" (moving the pieces manually using two boards) when competing with my program, so I can't test the relative strengths with many games (becomes too boring). But I did play through two games (and more will come, especially after new stuff has been implemented in either engine), and both became draw. The first game I decided to stop, since nothing was happening. Vicki was pawn up, but never did anything. Both just made meaningless moves, so at move 100 I decided upon draw, but Vicki had for the most part the better position. In game #2 there was a real draw by three-fold repetition.

Vicki is probably slightly better, but they are pretty equal.

Oh and btw, I've decided to call my chess program Babylon, influenced by my recent and valuable discovery of Babylon 5.

And now (drum roll), the games. (For the record, the time format was 3 6, but both used much less time that that.)

Round 1:



(There is some deep stuff going on in the game above. First, at move 36, Vicki sacrifices the exchange, and then just a couple of moves later Babylon also sacrifices the exchange. Both sacs seem unforced, but I may be wrong on that.)

Round 2:

Tuesday, July 03, 2007

Clash of the dilettantes -- game and quiescence search

Later in this post: a new game by my chess computer, against another chess computer. But first some boring techno-talk.

I've finally implemented what is called a quiescence search. It's a basic search algorithm that improves any chess computer greatly. I tried to implement it weeks ago, but there was a bug I couldn't find and I gave up knowing I would return to the task. Now I did, with success this time.

So, what's a quiescence search? A chess computer suffers from the horizon effect. It doesn't see beyond the assigned number of plies. So if it stops searching in the middle of a tactical sequence the evaluation of the position becomes very wrong. For example, white may just have taken a knight with its queen. Great, a knight up! Easy win! But, what the search algorithm didn't see is that black the next move can take the queen with a pawn. The computer didn't search deep enough to notice. But, that's with the ordinary search algorithm. Quiescence search to the rescue! Q-search is basically a search algorithm that searches only captures, so at the end of the ordinary search the q-search takes over and searches all the captures (as deep as needed), and evaluates the position as soon as no captures are left (technically it actually evaluates the position during the search too, for reasons I won't go into). No more searches that run out of depth in the middle of a tactical sequence.

Having a q-search makes the chess computer much more difficult to out-tactic (and it makes it more likely to out-tactic you!). And it is noticeable. My program was finally able to draw (with the better position) against its old nemesis The Thinking Machine. The Thinking Machine is basically a dilettante chess computer, but my chess computer isn't ready to take on Rybka just yet so I have go for easier competition for now.

The Game:



My chess computer (got to give it a name soon) is only thinking 3 ordinary plies deep (the q-search can be really time consuming) and was analyzing 34090 positions/s in average, and was thinking 3.11s per move average. It does play some funny moves, but not as funny or frequent as in earlier games. (It actually plays better now, with q-search, going only 1 ordinary ply deep, than it did before going 5 ordinary plies deep).

(And BTW, I was wrong in my post on null move pruning. It was wrongly implemented, and the branching factor and plies didn't get that boost. I'm not sure whether it improved anything at all, but it will when I get it right... but it's not a priority right now.)

Monday, July 02, 2007

TV watching

In the post on my favorite TV shows I halfly expected some critical comments for writing the following: "for years I foolishly lived without a TV". More exactly, for using the word "foolishly". That's because most people don't hold TV watching in high regard; it's a low status activity and many would consider it some sort of strength to live without a TV (even if they themselves don't).

I find this strange, because it so obviously equates "TV watching" in general with a certain kind of TV watching (although a common one). That's like saying that reading poetry or meditating is bad because it's possible to do it too much, or in a wrong way, or whatever. Anything can be done destructively.

There is also a light version of TV bashing, and that's when someone would say something like "sure TV watching has its uses -- it's great for entertainment or relaxation" with the obvious implication that while not being bad, TV watching still can't be, say, a deeply meaningful or spiritual activity, like I'm sure they would (if they're typical) say that reading books can be.

I'm suspicious of people whose value hierarchy happens to mirror the cultural norms exactly. It doesn't prove anything, but it indicates that they instead of independently having reached conclusions simply have adopted intellectual cliches, like "TV bad, books good". The indication becomes stronger when you get to know that they do watch TV but never read any books... (I might also add that my criticism is directed at TV bashing, not TV avoiding. It is a perfectly valid choice to not watch any TV. The reason it was foolish in my case is that it was the wrong choice for me. Or maybe it was the right choice at the time since I hadn't found the good TV stuff yet.)

But it's also worth saying that TV is a challenging medium to use correctly. It's easy to get too much of it (even watching boring stuff), and it's difficult to find the really good stuff (though somewhat easy to find *rather* good stuff, but that may not be good enough to spend time on.) I was serious when I said that it wasn't until I found Buffy the Vampire Slayer that I really understood the value of TV shows.

Anyway, having now seen four complete seasons of Babylon 5, I can easily add that show to the list of "deeply meaningful" TV shows. Really great show. Stay tuned for an update of my favorite TV shows list to see how I rate it in relation to my other favorites.

Friday, June 29, 2007

Chess computer and castling

You may have noticed in my last published game by my chess computer that none of the sides castled. That's no accident since I have no particular code to make it value castling (and in fact nothing to even make it understand when it's illegal -- it may, for example, move the king and then go back and then castle. Cheater!) That's because I haven't decided on how to do that.

It's not entirely trivial, for the following reason. You want the computer to value castling, but disvalue losing the ability to castle with a non-castling king move. So, you can't just have a single boolean variable that contains info on whether it has the possibility to castle and make your judgement based on that, because you haven't distinguished losing the ability to castle by castling, and losing it by just moving the king elsewhere, and you have to give points based on having the ability to castle or not having it. Which is best? There is no way to tell just from examining that variable.

At first I solved this by having two booleans. One called "can_castle" and one "have_castled". The default for them is of course "true" and "false", respectively. And then I gave plus for the combination have_castled = true and can_castle = false, and minus for the combination can_castle = false and have_castled = false, and neutral (no value either way) for can_castle = true and have_castled = false (the default). (Observe that the fourth combination -- can_castle = true and have_castled = false -- can never occur since having castled by necessity also means losing the ability to castle.)

Now I've decided to use just one variable, a simple int using the values 0,1,2 to cover the above possibilities. [Edit: 0-3 is better, so I can differentiate between short and long castling.] But I haven't implemented it yet. So I currently have nothing to tell the Ai to value castling specifically, which is why none of the sides castled in the recently published game. And that's that. (It does castle from time to time anyway, to achieve some other goal, like a more active rook.)

Edit: it didn't occur to me until now that there are cases when castling in one direction is all that ever will be allowed (when the castling rook has been moved). That has to be reflected in the code somehow too. Guess I have to use more numbers.

Wednesday, June 27, 2007

Chess computer: null move pruning

I've implemented a feature called "null move pruning" which helps the search algorithm to get rid of ('prune') more unnecessary variations. I'm down to an average branching factor of slightly above 8 (from about 11). That means it's now possible to (within reasonable time) go 6 half-moves deep. And that's reason to celebrate, like every new half-move. Not that long ago 4 half-moves was unplayable (took too long). Six is still not much, but it's better and it's something. More will come.

Now I'm starting to feel a need to be able to measure its performance in some more way than I'm currently doing. As long as I'm just speeding it up in terms of positions analyzed per second it's no problem since it can be measured directly and with little risk of error, but some other changes as not as easily measured (yet). Of course, counting half-moves (or branching factor), as I've done in the current case, is as good as any measure (or even better) as long as it's free of error, but I'm not sure I've implemented the null move pruning correctly and would like to have it verified by some other means. It seems stronger when I'm playing it, but it couldn't beat a chess program that it earlier (without the null move pruning) had a chance to beat (though it didn't). An accurate way to measure its rating would have been great. Maybe it's time to get it ready for FICS, or maybe download one of those packages with test-scenarios that I've read about. Or something.

Saturday, June 23, 2007

GTD -- what I'm still doing

Many GTDers "fall off the wagon" (as it's often called) from time to time. Not quitting completely, but getting sloppier, and the pre-GTD chaos comes back to some degree. It's easy to see why -- GTD is a comprehensive system with clearly defined actions and principles (with some room for tweaking and personalizing). If you adhere to some fuzzy system based on "do your own thing", it is almost impossible to "fall off" since by definition whatever you do is right. Not so with GTD, since it is an evil and cultish system.

I've never been a strict GTDer, but some things have stuck:

I do write down lots of things, and in a more orderly way than disorganized "to-do" lists. Not having to remember things in my mind is very relaxing, especially when the habit is so deeply set that you really trust your external system.

I write down ideas that comes to me, and I have different categories for different types of ideas. Sometimes I do things with these ideas, sometimes I just keep them to later reference. (Strict GTD would require making decisions about these notes more consistently.)

I write a lot of blog och forum comments, and keep a list of links to these posts. After a couple of weeks, sometimes more, of silence (no new comments) I erase that particular link with the assumption that the thread is dead. (Before I started to do this I always felt as if I was about to forget something, and sometimes I did.)

I keep a list of things I want to do with my chess program.

And much more. However, some of what I write down I don't look at again, at least not for a long while, making it almost useless.

And I don't keep all those elaborate lists with "next actions" with different "contexts" and so on. I do have a someday/maybe list, but I rarely read it. Still, knowing that everything I want to achieve someday is written down feels calming somehow. I know I will not wake up in 15 years suddenly remembering I should have achieved something during the latest 10 years. Maybe I won't achieve it, but not because I forgot about it.

The questions "what's the next action" and "what's the purpose" (in any given area) have been valuable, and it's something I ask myself a lot.

I have a much greater understanding of what causes chaos and stress -- lack of equilibrium between input (stuff coming into your life) and handling that stuff. Which means, among other things, that I now more easily can identify problem areas and work on them. (Still have a problem in this area though -- I tend to keep more stuff than I use or organize, especially non-physical stuff like bookmarks.)

And will also write a post on what software I still use. Some GTDers get obsessed with finding "the right software" to enhance their productivity. I didn't become like that, but I have tried (and discarded or kept) some. What you actually keep and discard is not always what you initially think when you try it out.

Thursday, June 21, 2007

Game played by chess computer

Played 2007-06-18, against itself.




4 plies (half-moves) deep.

time avarage per move: 0,613s
position analyzed per second, avarage: 30520
branching factor avarage: 11,0546037560704

Branching factor is how many moves it tries out in a given position. The avarage number of available moves during middlegame is something like 35. An unsophisticated search-algorithm (usually "minimax") tries them all out, whereas the better ones (always some version of "alphabeta") cut off lots of branches. How many branches it after pruning will try out depends on some features in its implementation -- it ranges from about 6 to 36. So my avarage branching factor isn't that bad compared to the worst case scenario. However, there is still a huge difference between 11 and 6. If you go 8 half-moves deep with a branching factor of 6, that's 1.7M moves to analyze, whereas 8 moves deep with a branching factor of 11 is 214M...

(It is possible to let it play 5 plies deep within reasonable time... about 7 seconds in avarage per move. Next game to post will be a game against Fritz, set to play at a lower rating, 5 plies deep.)

Saturday, June 16, 2007

chess program, funny position

Sir Edward Vs farm boy
It was finally dawning upon Sir Edward that he may lose.


Or maybe: "After careful consideration Sir Edward concluded he was worse off."

The above position came up when I was trying out a new method that sets up a position based on a string. Earlier I had few pre-coded positions in order to test the program in different positions, but the information of where to put each piece and pawn was integrated with the instructions that put it there. Now they are separated; one method handles the setting up, based on a string (which represents the position) sent as an argument. At first there was some problem and the position above came up (without the kings). Thought is looked funny. : )

I've started to measure the performance of the chess computer by letting it compete against other chess computers and record the result. I'll be back with some statistics. (It still plays badly, but it's fun to watch nonetheless, at least if you're the creator.)

Sunday, June 10, 2007

Delphi, hallelujah

Still love Borland Delphi. Very easy and intuitive to work with (which is important and leaves more time and energy to the intrinsic difficulties of a chess program, or whatever one is working with), and virtually bug-free. In all this time I still haven't encountered a single bug in the environment. It never hangs or crashes or anything. The chess program itself hangs all the time (because of temporal coding errors, like forgetting to increase the counter variable in a loop) but it never affects the environment within which I'm testing the program.

It is very easy to navigate through the code. The chess program currently exceeds 7000 lines of code, and it would be hopeless to work with without a good navigation system.

Some of the semantic features in Delphi are very useful. I've mentioned in an earlier blog post the possibility of starting an array with any index. I've now seen the usefulness of that. My board-representation was an 8x8 array. With a language that makes you start all arrays at zero, the board representation would range from 0 to 7, which is somewhat anti-intuitive compared to 1-8 (I also use constants a-h to represent 1-8, so to access e4 I simply type board[e,4], which is simple to work with. A language with zero-based arrays would use board[4][3] to access e4.)

But the real power of non-fixed starting index became obvious when I expanded my array to become a 12x12 array (in order to make checking for off-the-board coordinates easier.) With a zero-based array you would have rewrite the code so that board[3][3] becomes board[5][5] and so on for all earlier instructions (unless you've used constants, which I bet you haven't, you slacker). But with Delphi you simply make the new array range from -1 to 10 (assuming the old one ranged from 1-8), which means that the old board still applies in the exact same way. You don't have to rewrite a single line of code to make it work (although you do of course have to rewrite some part of the code in order to benefit from new board).

Okay, now I've spent too much time on a detail when the real strength of Delphi is how intuitive and easy to use it is (while still being powerful, as fast as C++ and so on). But to be honest my experience with other environments is somewhat limited, but I know many Delphi users agree with me. (For the record, I'm using Delphi 7 from 2002, so I don't know about other versions.)

Wednesday, June 06, 2007

chess program, board sweeping, inefficiencies (part 3)

No, not board sweeping of the good kind. And I realize these posts may not be of great interest for the majority of the readers, but since I'm evil I'm posting them anyway (insert maniacal laughter here).

There are a number of places in the program where it needs to have information on all the pieces. For example, when evaluating the position it needs to know what pieces are present on the board, and where (to take doubles pawns, placement of the pieces etc into account when evaluating). And when it checks for checks and for mate (and being in check) it also needs this info, as well as when it is generating all possible moves.

At this stage I 'sweep' the board every time. That is, this information isn't stored, so I go through the board square by square to collect this information. And this is done a lot; for every position evaluated (thousands each second), every time it generates all moves, checking for checks (which is done for each generated move).

I'm not going to do the math, but let's just say that it is sweeping the board *a lot*, and for no good reason since there is a simple way to cut out this operation -- by storing and updating this information as it changes with each new move. That means some adding some new instructions, but that's nothing compared to what I can remove. The net effect is increased efficiency.

And btw, with "checking for checks" for each generated move, I mean examining to see whether the ai will be in check after having performed the move, in which case the move is invalid (and will not be among the generated moves). However, that's fairly expensive, and a more efficient way is to skip that test and let it perform potentially invalid moves, and instead look for positions where the king is taken (which means that the previous position was invalid since it allowed for the king to be taken). [Edit: in fact, I've implemented this now, and it is considerably faster.]

Saturday, June 02, 2007

chess program, inefficiencies (part 2)

When I first implemented the rules (making the server/model -- that's what I call this part of the program, not to be confused with a web server -- recognize the difference between valid and invalid moves) I thought (probably correctly) I was going to have it ready a little sooner if I didn't have to make every method involved side-neutral, being able to handle both black and white. So, I made all the methods white-centric, and simply mirrored the internal board every time it was Black's turn, and then mirrored it back when the move had been examined (accepted or rejected). So: black makes a move, the board and move (and more) is mirrored, the move is examined as though it was White's move and White's turn, then everything is mirrored back, and the move is either executed (if accepted) or not (if not).

That may not sound like much work, after all computers are pretty fast, yes? Well. It isn't, and they are. But when the chess computer is thinking, it is trying out a lot of moves. And every time it's Black's turn to move, that routine is performed (and every time it's White's turn, it has to run several tests to see whose turn it's, to know whether it should switch or not, so computing power is used even when it doesn't switch). It's an essential part of the process, and will be done many times. To make matters worse, I found it to be fairly easy to make those white-centric methods side-neutral (which also takes some computing power, testing for side...), so I implemented that too on top of the mirroring (but not exactly everywhere, so I couldn't cut out off the mirroring...). So I handled the same problem twice, wasting CPU cycles.

That mirroring, which is long gone now, isn't the most interesting example of inefficiency, but illustrative of a general principle in how to gain computational efficiency: to make the computer do as much as possible with as few (and cheap, time-wise) instructions as possible. Referring here to the number of instructions it actually runs, not how many instructions are written in the code (so a loop with three written instructions running 100 times, is counted as 300 instructions, not three.) (In the final analysis the instructions to be concerned about are the ones on assembly-code level, or even machine-code, however that would be waaay overkill at this stage, and perhaps in all future stages as well.)

That principle is btw similar to the GTD principle of trying to get as much as possible done with as little effort as possible, and the scientific principle of trying to explain as much as possible with as few principles as possible.

Thursday, May 31, 2007

chess program, methodology, inefficiencies (part 1)

I haven't spent much time planning in the process of writing the chess computer, but rather have been working by creating increasingly more well structured (and efficient) prototypes. As against extensive planning to get everything right the first time you get around to coding. It's not as chaotic as it sounds, I do make decisions regarding structure along the way, just that I do it when it comes naturally, rather than trying to force it. And I try to adhere to design principles such as low coupling och high cohesion. But still, as soon as planning isn't easy or obvious I start to code until my insight into the program is such that I'm ready to for new decisions regarding structure (which often means some degree of refactoring needs to be done).

As a result of this, my chess program has gone through some stages when my code was very different (in some ways) from what it is now, and less efficient and less well structured (not that it's particularly efficient now, but everything is relative and all that). I just thought I'd document some of these inefficiencies for future reference. Starting in my next post (I can't keep my plan to post three times a week if I write those long posts every time...)

(Is there any way to always be logged in at blogger? It annoys me having to log in every time, even if it only takes a click.)

Tuesday, May 29, 2007

Positional elements and what to do

It is not entirely obvious what various positional elements mean in terms of actions. Not to be me anyway. It is quite possible to be able to assess a position, see who's got the space, the well places pieces and so on, yet not know what to do, neither in terms of goals nor next move. (And I'm not speaking of breakthrough positions... in an general, positional way you do know what to do in those, achieving the breakthrough.)

One of my goals, one that I think is of crucial importance in playing chess, is to always have some grasp of what to do in position. So when I blitz, as I think I've explained, I'm always on the lookout for positions in which I'm clueless, and after the game I think about what I should have done (in terms of positional goals, usually not specific moves). I also think about what my opponent does, especially in cases when my opponent is in a position that I don't know what I would have done in. More than once I've learned something about what to do just by observing what they did, and their ensuing success.

This and tactics is the way to improve, I think.

Much of this is tightly related to things I've already written about, but it's not exactly the same.

Here's a list of very general plans and goals related to a certain positional elements: (in all cases it is also desirable to prevent the opponent from achieving these things, but I write primarily from the perspective of what to do yourself with your position).

Opening: develop pieces, control center

Endgame: promote pawns

Weak (enemy) pawns: attack it

Weak (enemy) square: put a piece on it.

Pawn majority: push them forward

Space advantage generally: use it to manoeuvre pieces (attacking something, I suppose.)

Space advantage on kings side (normal castling): attack the king (if enough pieces left)

Badly placed piece: activate it

(I'm not too happy with these examples, there should be more tricky ones. Maybe I'll add to the list as soon as I've come up with more.)
---

The general point is to always to able to translate static qualities into what it means in terms of action. Needless to say, most positions have several of these at once, and just because you have a badly placed pieces doesn't mean your immediate goal should be to activate it. Maybe something else more important at that moment. Still, when you see that piece your mind should automatically make the connection that it would be nice to have it more active. That's an obvious example, but it's not always that obvious. We all have 'blind spots', and for me one particular blind spot is pawn majority. I've been in blitzes not knowing what to do and afterward found it obvious that I should have pushed those pawns. The solution is to keep repeating that connection until I see it without effort.

"Every is implies an ought" (Ayn Rand)

Friday, May 25, 2007

Positional elements, analysis and integration of

First a short announcement: I've finally created a link list. Look to the left and you shall find. If someone linking to me isn't added, let me know and I'll consider including you (which I most likely will, unless you write really wicked stuff).

Okay, it's time to go mendelejev again, this time to identify groups of positional factors (not to be confused with the semi-positional elements in the tactical inventory, that I've written about else).

What I once learned and retained is the following, these four groups:

1. Safety of the king
2. lines and diagonals (lines includes rows)
3. squares and pawns (including weak pawns/squares, pawn-formation, etc)
4. Placement of the pieces.

Edit: actually, I haven't retained it correctly after all, since "space" must be, and was, included. Yet I'm sure there were exactly four groups. Maybe space belongs with "placement of the pieces"? No... um. Whatever, let's add a fifth category until further notice:

5. Space

(Edit 2: and what about the center, shouldn't that be mentioned? Uhmm.)

Does that cover it all?

I thought I had got it from "The Art of the Middlegame" (Keres/Kotov), but I've searching in that book and can't find it. Perhaps it was from "Thinking like a Grandmaster". Anyone know? Or maybe I just compiled the list myself, using these works. Don't remember, although I do remember Keres (or Kotov) specifically saying something about the value of extracting the various elements, specifically comparing the activity to creating the periodic system.

Ayn Rand identified four elements in fiction: plot, characterization, style and theme. She pointed out that these parts are not separate in the fiction itself, it's just in our mind that we can pick out the parts and analyze them by themselves. For example, through the character's actions we learn both about the character himself (characterization) and it also drives (or should do) the plot forward, and it's also part of the theme and expressed in a certain style, and so on.

Likewise in chess. Having weak squares around the castled king both falls under "safety of the king" and "squares and pawns", and possibly under "placement of the pieces" (you would like a black knight on h3, wouldn't you?) as well as lines and diagonals (like that killer bishop on b7 that is just about to shoot the white king in the head). These things are picked apart intellectually only, in reality they are connected (and they should be connected in our minds too, part of the process of analysis is to discover these connections. So analysis has a close relation to integration and 'holism', actually. Or should have.)

To be continued. I've already written the next part (actually I haven't, but I thought I would by the time I published this, but I will), which is about connecting positional analysis with goals. What exactly do we *do* with these positional elements after they've been identified? In GTD terms, what's out purpose and what's the next action?

(I like the phrasal verb to "go mendelejev", and I think I'll make "mendelejevism" its own category/label. Good thing about not having english as your first language; you get to say anything, including making up your own words and phrases. : ))

(And btw, I've now seen the first season, and 1/3 of the second, of Babylon 5, and I like it a lot. I'll have more to say about that.)

Thursday, May 17, 2007

Chess -- chess computer, some info and a game

I've recorded another game played between me and my chess computer, this time it's thinking four plies (half-moves) ahead so it plays a little better tactics-wise, but overall still pretty badly. But eventually it will start beating me so I should relish these winning days 'cause they won't be here again.

Actually I haven't tried to make it any stronger since last time (except that I now have the Alphabeta search algorithm, and Negascout is on its way). It's the same algorithms and evaluators. What I've spent my time doing is adding things and re-structuring the code. I've now (mostly) implemented it using the MVC design pattern. It has some nice advantages I might be writing about some time. Also, I've added graphics (or maybe I've already said that?). Earlier I had a board with labels showing numbers, like this:




Now it looks like this: (I still have plenty to do, but there's a difference.)



As you may notice, I've used the graphical depiction used on CTS (using screen shots). I contacted them twice about this, asking for permission. They didn't respond. Perhaps any reader knows whether it would be a copyright violation to use this board and these pieces. I mean, chess pieces are really trivial graphics (especially these pieces -- they are the 'standard' pieces), so it's almost like asking for permission to use a black background, but if there is a problem I'll create my own or get some other.

Here's the game:





It unexpectedly won a pawn in a quiet position, but other than that it really needs to read (and study!) Keres, Euwe and the others. But you know how they are, today's chess computers. It's all about beer, chicks and crappy TV shows. Back in the days...

Monday, May 14, 2007

GTD, chess and Objectivism

There are some parallels between these. Ayn Rand considers the three cardinal values in life to be reason, purpose and self-esteem. In GTD the the two big questions are what you want to achieve (or in other words, what's your purpose?) and what's the next action. So purpose plays a big part in both these. Also true in chess, except that 'goal' is a more commonly used term, and in chess there's also a correspondence to GTD's 'next action', which of course in chess simply is 'next move'. Objectivism doesn't have a direct correspondence to 'next action', but then it's a philosophy and that would be too narrow a concern. That's why Objectivism and GTD work so well together -- Objectivism identifies broad principles (including that productivity is a virtue) and GTD is a system that applies parts of Objectivism (such as being more productive) to your dialy life. That's how I see it anyway, and I know GTD is somewhat popular among Objectivists (and would be more so if people knew about it).

As for chess. I think of chess as a magnifier of these and other principles, perhaps especially purpose and causality. Living is often ambiguous and bewildering. You don't know what you want, or how do get it, or what follows, or the identity of certain objects. Chess is much more manageable. Much of what makes life bewildering and complex is removed, yet those really important principles in life remain in chess, writ large. You still want things in chess, and things follow, and things have identity. Chess creates a sense of order and clarity, a much needed feeling.

Friday, May 11, 2007

Chess -- Another breakthrough position

Here's another breakthrough position, one that I got from Susan Polgar's blog (a blog every chess player should read, but then I guess most already do) the other day:



Notice the elements making it a 'before the breakthrough'-position: white has the superior position, all pieces are well placed (any positional move seems to just make the position worse), the king is safe. It's an open position, so making pawn moves are out of the question. And you can't have a positional goal such "making a piece more active" either, because all pieces are already fully active. Vintage pre-breakthrough; something radical is required.

In this position the solution happens to be purely tactical (nice tactics, find it out), which in a way is the least interesting type of breakthrough, since tactics isn't as ambiguous as other ways to continue, like sacrifices or 'making threats and switching targets'. (I need a better name for the last one.)

Monday, May 07, 2007

This just in: crazy game by computer

Okay, here's a game my chess computer just played (both sides). It's thinking only two plies (half-moves) ahead, less than 0.5 s per move. It evaluates a position according to three criteria: material worth (of course), center squares, and a secret criteria. It knows no opening theory, of course, and there is no randomizer, so it plays this game every time (given two half-moves. It would play better looking ahead more moves, but I'll wait until I've implemented the alphabeta pruning algorithm before I play through any longer games, as well as some other changes.)

I've now added a nice looking board and some real pieces (earlier I just had board with numbers representing pieces). And I've increased the search speed to at least 6000 positions per second. Still haven't put any real effort in speed increasing, so I expect to gain more later. Anyway, the game:




I actually don't understand why it plays some of the moves, but then my third criteria is difficult to calculate as a human (and it probably makes it play some bad moves, but I had to throw in something quickly in order to make it stop playing a3 and then Ra1-Ra2 back and forth...)

.

Saturday, May 05, 2007

TV shows, top 8

Update: a more current (but to a large degree similar) TV show top list has been published here. Aside from being more current it also contains a couple of nice pictures and a link to info on every show.

Being a great fan of TV shows I thought I'd try to compile a list of my favorite shows. Kind of a recent interest actually, for years I foolishly lived without a TV. Then I changed. It all began with Buffy the Vampire Slayer...

In order:

1. Buffy the Vampire Slayer
2. Angel
3. Here it is starting to get tricky. I let these shows share the third spot, without order:

House M.D.
Veronica Mars
Firefly

4. Battlestar Galactica

5. No order:

Star Trek: the Next Generation
Lost

There are other good TV shows, but not good enough to mention. Next show to explore is Babylon 5. I recently bought season 1-3 and The Gathering.

Thursday, May 03, 2007

Chess -- positional play and pawn structure

One of the most important factors to consider when deciding upon a positional goal is pawn structure. And I've found the best ever site to explain what that's all about. This one. Apparently based on a book by Euwe and Kramer, currently out of print.

.

Monday, April 30, 2007

Chess -- positional analysis of a blitz

Today a quick analysis (not analysis as in 'calculation') of a recent blitz (3 10) at FICS. Not that it's a particularly profound or interesting game (and I have barely looked for missed tactics with a chess computer, but in this analysis I'm more interested in ideas and position than exact lines anyway), I just feel that I have something to say about it. It's a Caro-Kann, which is my preferred opening as black in response to 1.e4.

First some opening moves, then this position:



In my mind, the next positional goal for each side is pretty clear: White wants to play f5 and go for an attack on the black king. Black wants to play c5 and take on d4 (opening the line, and also making the pawn on e5 pretty weak in case white plays f5.)

In these positions with each one playing on each side of the board, speed is important. The one breaking through first can hope to force the other one to defend, getting the initiative. (As long as no one is forcing the other to defend, no one has the initiative the way I see it, but I've seen this view disputed.) So, with that in mind my next move in the position above was a6. The idea was to prevent Nb5 and Nd6. I didn't have time to calculate whether that manoeuvre is actually playable or meaningful. However, I think a6 is bad and that it's better to just capture the knight (Bxc3) and then play c5 (or c5 directly if I deem Nb5-Nd6 to be of no threat).

Then white played a3, I responded with Bxc3 and white played Pxc3.

Then I made another, much graver, mistake. Kind of a positional suicide. Here is the position before the mistake. See a good way to play positionally badly?



I should have played c5 here. In case of dxc I can't retake the pawn immediately, but it's undefendable, so I can then play Rc8 and hopefully start getting some play on the c-file.

I played Nb6. The idea being, of course, to play it to c4. Which makes a nice outpost, however it's bad because now c5 isn't playable anymore, and without that the Knight on c4 is pretty much all I get on the queen side, and I will have to start to seriously worry about the safety of my king, because now white played f5, at last. And it's a killer. Pawn takes, knight takes, knight takes, rook takes. Then we have the following position. Black's king is starting to get really scared:



Black to play. How to get out of this mess? The Bishop will take on h6 at any moment. Looks lost. The only move I can think of is Qh4, getting some extra defence against the attack (white won the race and has the initiative), hoping to exchange queens.

I did play Qh4, and white played the natural Raf1. Black then played the also natural Nc4, finishing that manoeuvre. White played Bf2 threatening black's Queen. Positional mistake though, allows for Qe4 (which I played) almost forcing an exchange of queens. And he did play Qxe4, I took with the pawn and he played Re1 (going to take the pawn). I can't defend it, so I take the pawn on a3, Nxa3, and white played Rxe4. The attack on black's king failed (or is it any life left in it?) and now the game takes on a different character once again, so it's time for a diagram (black to play).



I can't say who has the best position at this point (GNU chess prefers white despite the hanging pawn on c2), but I do know that I by far prefer black's position. Why? Because it's easy to play. The positional goal is clear: running with the a-pawn to promote it, while fending off whatever white is going to try to do. What should white try to achieve positionally? I'm not sure but maybe marching onwards with the center pawns (which is what happens), maybe even going for an attack on black's king.

Btw, black shouldn't waste any move on capturing the pawn on c2. I played a5.

Some more moves where made. White played rather aimlessly (as I too would have as white in that position, I think) and managed to lose both center pawns in a desperate attempt to create some threats. Eventually this position came up (white to move):



White is lost, of course. I managed to get my rook now on d7 to b1, and won.

Here's the complete game. (Is it possible to switch side on these boards, to see it from black's perspective?)




The End.

Friday, April 27, 2007

Chess Computer 2

I've fixed the memory leak. Turns out that it was caused by not overriding the destructor (not something needed to do in other languages I've used, that I recall). Some of my objects have their own internal objects, and they are de-allocated in the destructor, so when it wasn't called they weren't, and they quickly became numerous since it's a type of object that I create and destroy a lot when the ai is thinking (hundreds of MBs became allocated if I left the Ai thinking for a shore while...) I spent hours trying to find that leak. No warning from the compiler or anything (it warns about all sorts of things, like declaring a small variable and then not using it, but if you write the worlds longest and coolest destructor without overriding it, it won't say a peep. Boo, etc. But still, Delphi's great.) Anyway, now that's fixed, although I still have a minor memory leak yet unfound.

The ai examines about 2000-3000 positions per second. That's pretty bad, but I don't mind as I've not taken speed into account when writing the code. It will get better as I refactor the code. (My main consideration has been the structure of the code, at least on a higher level like what objects and what relations etc, but that too will be improved.) At first I got the result 4000-5000 positions per second, and even 10000, but then it got lowered. I'm not sure why, but I've changed the code back and forth without keeping track of anything. Anyway, I like things measurable and it will be fun to see what speed I can reach. And with a baseline like this, it can only get better.

Apart from being slow in general, the search-algorithm is also ineffective in another way. It searches all branches, even useless ones. I've implemented the minimax algorithm so far, and my next step is using alfabeta pruning. Code-wise the change is minimal, but search-wise the change is huge. The same number of positions will be examined given the same time, but irrelevant branches will be cut off. And without risk too (as compared with minimax, but of course both suffers from the horizon effect, but alphabeta normally sees further given the same time.)

.
 
Locations of visitors to this page