CRL Task 1: Generalised Policy Learning

In the previous blog post we developed some ideas and theory needed to discuss a causal approach to reinforcement learning. We formalised notions of multi-armed bandits (MABs), Markov Decision Processes (MDPs), and some causal notions. In this blog post we’ll finally get to developing some causal reinforcement learning ideas. The first of which is dubbed Task 1, for CRL can help solve. This is Generalised Policy Learning. Let’s begin.

This Series

  1. Causal Reinforcement Learning
  2. Preliminaries for CRL
  3. CRL Task 1: Generalised Policy Learning
  4. CRL Task 2: Interventions – When and Where?
  5. CRL Task 3: Counterfactual Decision Making
  6. CRL Task 4: Generalisability and Robustness
  7. Task 5: Learning Causal Models
  8. (Coming soon) Task 6: Causal Imitation Learning
  9. (Coming soon) Wrapping Up: Where To From Here?

Generalised Policy Learning

Reinforcement learning typically involves learning and optimising some policy about how to interact in an environment to maximise some reward signal. Typical reinforcement learning agents are trained in isolation, exploiting copious amounts of computing power and energy resources.…

By | July 1st, 2021|Background, English, Level: intermediate|4 Comments

Preliminaries for CRL

In the previous blog post we discussed and motivated the need for a causal approach to reinforcement learning. We argued that reinforcement learning naturally falls on the interventional rung of the ladder of causation. In this blog post we’ll develop some ideas necessary for understanding the material covered in this series. This might get quite technical, but don’t worry. There is still always something to take away. Let’s begin.

This Series

  1. Causal Reinforcement Learning
  2. Preliminaries for CRL
  3. CRL Task 1: Generalised Policy Learning
  4. CRL Task 2: Interventions – When and Where?
  5. CRL Task 3: Counterfactual Decision Making
  6. CRL Task 4: Generalisability and Robustness
  7. Task 5: Learning Causal Models
  8. (Coming soon) Task 6: Causal Imitation Learning
  9. (Coming soon) Wrapping Up: Where To From Here?

Preliminaries

As you probably recall from high school, probability and statistics are almost entirely formulated on the idea of drawing random samples from an experiment. One imagines observing realisations of outcomes from some set of possibilities when drawing from an assortment of independent and identically distributed (i.i.d.) events.…

By | April 6th, 2021|Background, English, Level: Simple|5 Comments

Causal Reinforcement Learning: A Primer

As part of any honours degree at the University of Cape Town, one is obliged to write a thesis ‘droning’ on about some topic. Luckily for me, applied mathematics can pertain to pretty much anything of interest. Lo and behold, my thesis on merging causality and reinforcement learning. This was entitled Climbing the Ladder: A Survey of Counterfactual Methods in Decision Making Processes and was supervised by Dr Jonathan Shock.

In this series of posts I will break down my thesis into digestible blog chucks and go into quite some detail of the emerging field of Causal Reinforcement Learning (CRL) – which is being spearheaded by Elias Bareinboim and Judea Pearl, among others. I will try to present this in such a way as to satisfy those craving some mathematical detail whilst also trying to paint a broader picture as to why this is generally useful and important. Each of these blog posts will be self contained in some way.…

By | February 3rd, 2021|Background, English, Level: intermediate, Level: Simple|5 Comments

Using integration to calculate the volume of a solid with a known cross-sectional area.

Hi there again, I have not written a post in while, here goes my second post.

I would like us to discuss one of the important applications of integration. We have seen how integration can be used to solve the area problem, in this post we are going to see how we can use a similar idea to solve the volume problem. I suggest that we start by looking at the solids whose volume we know very well. You should be able to calculate the volumes of the cylinders below (yes,  they are all cylinders.)

 

circular cylinder                                 rectangular cylinder                triangular cylinder

Cylinders are nice, we only need to multiply the cross-sectional area by the height/length to find the volume. This is because they have two identical flat ends and the same cross-section from one end to the other. Unfortunately, not all the solid figures that we come across everyday are cylinders. The figures below are not cylinders.…

Mathematics societies in Africa – very little information on wikipedia

If you go to the Wikipedia page for the list of mathematics societies, you will see that very few societies within Africa are shown. A very quick Google search shows that many countries do indeed have their own mathematics society, so it would be great to try and find out which ones are active. If you know of an active society for a country which is not in the list (currently only South Africa and Gabon are listed as far as I can see), then please write it in the comments, we can collate them, and upon contacting them and asking whether they want to be listed on the wikipedia page, we can put in their details. Please spread the word and let’s see if we can get a few more societies highlighted.

How clear is this post?
By | June 12th, 2015|Background|4 Comments

Mathemafrica poster – please spread the word!

Our very talented web developer Adam has designed a poster to advertise Mathemafrica.org. If you are in a mathematics department anywhere within Africa, or a highschool where you think this might be of interest, please think about printing the poster out and putting it up. If you would like to translate the poster into another language, we would also very much appreciate your help!

MatheM_poster

Download the poster here.

How clear is this post?
By | June 10th, 2015|Advertising, Background, English|0 Comments

National Conference on Multilingualism in Higher Education

Eventually we want to make this a multilingual blogging platform. Some of the translations of the framework are in, and we hope to have the structure implemented soon. In addition we would love to get people blogging in other languages, and to get some of the current content translated.

The multilingual nature of South Africa is a very important issue in terms of education and there will be a conference focusing on this in August at UNISA. Check it out here.

How clear is this post?
By | June 4th, 2015|Background, Conference advert|0 Comments

The Varsity Maths Problem

The following post is written by John Webb from The Department of Mathematics and Applied Mathematics at The University of Cape Town. With his permission I include it here as an advert for a book which is discussed at the bottom of the post. Mathemafrica receives no payment for including this text. I hope that in addition to being an advert for the book, this may be a chance for students to discuss some of the problems they see with the transition between school and University here in South Africa for maths students.

Why do so many first-year students fail varsity maths?
Thousands of students across South Africa have started their university careers, and many of them have enrolled for a course in Mathematics. Some will be aiming at a maths major, in particular those who hope to teach mathematics at school level. But far more will be doing maths as a requirement for their degrees in a whole range of areas.…

By | June 3rd, 2015|Background, English, Level: Simple|4 Comments

Mathematics or dreams, which is more real?

Mathematics can sometimes seem dream-like, at least on first encounter. Later on, one gets
used to a new mathematical object, and it seems everyday. I remember how strange the idea of
a group was to me, how mysteriously it grew from three almost trivial axioms to a forest
of subgroups and quotient groups and equivalence classes and so on. Of the few dreams I
now recall, there was one with a huge hall full of people, perhaps a giant cave, and I was descending a long,
rickety staircase — or was I sliding down a cable? — feeling myself among a heretofore
completely unsuspected part of humanity, who perhaps nobody from above ground had ever seen.
Groups were a bit like that, and saying that a square had a symmetery group did not
make them appear any less unexpected.

Furthermore, dreams and mathematics have a lot in common—I mean here the dreams that
people, when awake, remember having had when asleep.…

By | April 13th, 2015|Background, English, Level: Simple|2 Comments

Mathemafrica needs You!!

We would love to develop the content here, but for that we need your input!

Whether you are a first year university student starting to study varsity level maths, or you’re a PhD student working on your thesis, a lecturer who has many years of research behind you, somebody working in mathematics education, a high school student who has suddenly come across a particular idea which blew your mind, or perhaps confused you to no end, we would like to hear from you.

We would like people who feel inspired to write about what mathematics means to them, and the parts of mathematics which inspire them.

Importantly we would like diversity both in terms of the level of mathematics discussed here as well as the language used. So whether you’re an English speaker, a Shona speaker, a Xhosa speaker, an Arabic speaker or any other language speaker/writer, please feel free to write in whichever language you are most comfortable.…

By | February 18th, 2015|Background, English|0 Comments