{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "cell-000",
   "metadata": {},
   "source": [
    "---\n",
    "title: Character-Level RNN on WikiText-2\n",
    "category: Deep Learning\n",
    "tags:\n",
    "  - deep learning\n",
    "  - rnn\n",
    "  - gru\n",
    "  - language modeling\n",
    "  - wikitext\n",
    "  - pytorch\n",
    "date: 2026-07-31\n",
    "description: Train a small recurrent language model on WikiText-2 and inspect how sequence state evolves.\n",
    "notebook_page: false\n",
    "aliases:\n",
    "  - /deep-learning/character-level-rnn-on-wikitext-2\n",
    "sources:\n",
    "  - https://huggingface.co/datasets/Salesforce/wikitext\n",
    "  - https://docs.pytorch.org/tutorials/beginner/nlp/sequence_models_tutorial.html\n",
    "---\n",
    "\n",
    "## Goal\n",
    "\n",
    "Train a small character-level recurrent model on [WikiText-2](https://huggingface.co/datasets/Salesforce/wikitext). We will crop the corpus down to a few hundred thousand characters so the notebook stays laptop-sized while still learning visible local structure.\n",
    "\n",
    "For a recurrent network,\n",
    "\n",
    "$$\n",
    "h_t = f(W_{xh} x_t + W_{hh} h_{t-1} + b_h), \\\\qquad\n",
    "\\\\hat{y}_t = W_{hy} h_t + b_y.\n",
    "$$\n",
    "\n",
    "We will use a GRU because it keeps the code compact while still preserving the recurrent inductive bias.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "cell-001",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-07-31T21:59:00.767510Z",
     "iopub.status.busy": "2026-07-31T21:59:00.767397Z",
     "iopub.status.idle": "2026-07-31T21:59:02.694133Z",
     "shell.execute_reply": "2026-07-31T21:59:02.693509Z"
    }
   },
   "outputs": [],
   "source": [
    "## %pip install -q datasets torch matplotlib\n",
    "\n",
    "import math\n",
    "import random\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "from datasets import load_dataset\n",
    "\n",
    "torch.manual_seed(7)\n",
    "random.seed(7)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "cell-002",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-07-31T21:59:02.696092Z",
     "iopub.status.busy": "2026-07-31T21:59:02.695856Z",
     "iopub.status.idle": "2026-07-31T21:59:04.704933Z",
     "shell.execute_reply": "2026-07-31T21:59:04.704295Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "chars=300,000 vocab=164\n"
     ]
    }
   ],
   "source": [
    "ds = load_dataset(\"Salesforce/wikitext\", \"wikitext-2-raw-v1\")\n",
    "train_text = \"\\n\".join(ds[\"train\"][\"text\"])[:300000]\n",
    "valid_text = \"\\n\".join(ds[\"validation\"][\"text\"])[:50000]\n",
    "\n",
    "vocab = sorted(set(train_text + valid_text))\n",
    "stoi = {ch: i for i, ch in enumerate(vocab)}\n",
    "itos = {i: ch for ch, i in stoi.items()}\n",
    "\n",
    "train_ids = torch.tensor([stoi[ch] for ch in train_text], dtype=torch.long)\n",
    "valid_ids = torch.tensor([stoi[ch] for ch in valid_text], dtype=torch.long)\n",
    "\n",
    "print(f\"chars={len(train_ids):,} vocab={len(vocab)}\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "cell-003",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-07-31T21:59:04.706482Z",
     "iopub.status.busy": "2026-07-31T21:59:04.706334Z",
     "iopub.status.idle": "2026-07-31T21:59:04.711011Z",
     "shell.execute_reply": "2026-07-31T21:59:04.710470Z"
    }
   },
   "outputs": [],
   "source": [
    "def get_batch(data: torch.Tensor, batch_size: int, block_size: int, device: str):\n",
    "    starts = torch.randint(0, len(data) - block_size - 1, (batch_size,))\n",
    "    x = torch.stack([data[s : s + block_size] for s in starts]).to(device)\n",
    "    y = torch.stack([data[s + 1 : s + block_size + 1] for s in starts]).to(device)\n",
    "    return x, y\n",
    "\n",
    "\n",
    "class CharRNN(nn.Module):\n",
    "    def __init__(self, vocab_size: int, embed_dim: int = 64, hidden_dim: int = 128):\n",
    "        super().__init__()\n",
    "        self.embed = nn.Embedding(vocab_size, embed_dim)\n",
    "        self.rnn = nn.GRU(embed_dim, hidden_dim, batch_first=True)\n",
    "        self.proj = nn.Linear(hidden_dim, vocab_size)\n",
    "\n",
    "    def forward(self, idx, hidden=None):\n",
    "        x = self.embed(idx)\n",
    "        out, hidden = self.rnn(x, hidden)\n",
    "        logits = self.proj(out)\n",
    "        return logits, hidden\n",
    "\n",
    "    @torch.no_grad()\n",
    "    def generate(self, idx, max_new_tokens: int):\n",
    "        hidden = None\n",
    "        for _ in range(max_new_tokens):\n",
    "            logits, hidden = self(idx[:, -1:], hidden)\n",
    "            probs = logits[:, -1].softmax(dim=-1)\n",
    "            next_idx = torch.multinomial(probs, num_samples=1)\n",
    "            idx = torch.cat([idx, next_idx], dim=1)\n",
    "        return idx\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "cell-004",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-07-31T21:59:04.712403Z",
     "iopub.status.busy": "2026-07-31T21:59:04.712307Z",
     "iopub.status.idle": "2026-07-31T21:59:38.673821Z",
     "shell.execute_reply": "2026-07-31T21:59:38.673172Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "step=0000 loss=5.1217\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "step=0200 loss=1.9348\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "step=0400 loss=1.7522\n"
     ]
    }
   ],
   "source": [
    "device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n",
    "model = CharRNN(len(vocab)).to(device)\n",
    "optimizer = torch.optim.AdamW(model.parameters(), lr=3e-3)\n",
    "\n",
    "batch_size = 64\n",
    "block_size = 128\n",
    "train_losses = []\n",
    "\n",
    "for step in range(600):\n",
    "    xb, yb = get_batch(train_ids, batch_size, block_size, device)\n",
    "    logits, _ = model(xb)\n",
    "    loss = F.cross_entropy(logits.reshape(-1, logits.size(-1)), yb.reshape(-1))\n",
    "    optimizer.zero_grad()\n",
    "    loss.backward()\n",
    "    torch.nn.utils.clip_grad_norm_(model.parameters(), 1.0)\n",
    "    optimizer.step()\n",
    "    train_losses.append(float(loss))\n",
    "\n",
    "    if step % 200 == 0:\n",
    "        print(f\"step={step:04d} loss={float(loss):.4f}\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "cell-005",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-07-31T21:59:38.675373Z",
     "iopub.status.busy": "2026-07-31T21:59:38.675167Z",
     "iopub.status.idle": "2026-07-31T21:59:38.748704Z",
     "shell.execute_reply": "2026-07-31T21:59:38.748223Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAArMAAAGICAYAAABSsmjoAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjExLjEsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvctoD+AAAAAlwSFlzAAAPYQAAD2EBqD+naQAAZihJREFUeJzt3QdcU/cWB/DDXoLIUhFQBPdW3HtW66yjVq1VW61aq7XD1tFWW31qrV0+6+vQatWq1da6te699wQHDlRkOBgiO+9zDt6QQNCAISHw+34+KcnNTXK5UDw5Of9zLFQqlYoAAAAAAMyQpakPAAAAAAAgrxDMAgAAAIDZQjALAAAAAGYLwSwAAAAAmC0EswAAAABgthDMAgAAAIDZQjALAAAAAGYLwSwAAAAAmC0EswBgMhs3bqR169bhJ1CAfh4bNmww2eNflKlfHwBMwwITwAAgPzx48ICOHj1KUVFR5ObmRuXKlaOqVauShYWFep9WrVpRYmIiHT582GQ/hO3bt9ONGzfUt+3s7MjX15caN24s13Pav3z58tSmTZts9y9ZsoRKly5N7dq1e6HH6MKBv5WVFXXu3JnyA/88UlNTaf/+/SZ5/Isy9esDgGlYm+h1AaCQ4uD1vffeo7/++ovq1q1LFSpUoNjYWNq9ezeVLFmSpk6dSn379qWCYu7cuRIkvvnmm3I7JiaG9uzZI0H3H3/8kS3A5P3Xrl1LTk5OdPXqVSpVqpTW/aNHj5agSvNxeXmMLtOnTyd7e/t8C2b5edPT0032eACAvEAwCwAGEx0dTQ0bNiRLS0s6fvw41axZU31fQkICffrpp7RmzZoCFcwyPt758+erbz98+JDq1KkjxxkWFkaOjo5a+9vY2FBKSgpNnjyZfv75Z71eIy+PMbZx48aZ9PEAAHmBYBYADOb999+nW7du0ZkzZ6hatWpa93FA+O2338p9uty+fVs+Hub9+KP4YsWKad1/4MABunTpkjr4LFGiBDVo0IDKlCmTrW4yLS2NunXrJs958OBBKXN4XtZTEz/3q6++Sl9//TUdO3aMWrZsme17GTJkCP33v/+V77ly5crPfc68PEbTsmXLJOttbW2tDrz5Ofv37y/X//nnH7n90ksvSUkDl254e3tTixYtaNeuXXTt2jX1uXN3d6dGjRpJpjzruVOpVNSlSxf1Ns3nvX79ujyvi4uL/IwcHBwM+njGAT9n8SMjI6l27drye8S3w8PDqV+/fpRX/OYqODhYMtv8hotLSbIKDQ2ls2fPUnJyMlWpUoVq1KiRp30AwLgQzAKAQXApwYoVK6h9+/bZAllNtWrVyrZt8eLF9MMPP1BgYKAEtPwR/759+8jf31+9jxIIKQEPB2d8e/z48TRt2jT1fhyAch3uvXv3aN68efIcXPuam2CWKfWy/Fq6fPbZZ7Ro0SL65JNPpIRAH3l5jOLEiRMUFxcnNbPKeShevLg6mP3iiy+kfOHcuXO0fPlyCfI5WOVg9sqVK1K/zDgICwkJoZMnT0rZgmY2lc8d15xqBqPK83J5BB87n08OLp2dneUNhmbJxIs+ngPFTp06Sb01Hzf//Jo0aSKBLD8mL8HsnTt3qHfv3vJGiEs5OOvOz/Puu+/SN998I8E9e+edd+T4eB9XV1eaMWOGnN+///5bgn999wEAE+AFYAAAL2rnzp0q/pPy+eef6/2Yli1bqkqXLq366KOPVOnp6bItMjJSVaJECdUbb7zx3Mf/9ddf8pp79+7Ves5SpUqp3n33XfVz3rp1K8fn6N69u8rKykprW1pamqpevXoqe3t7VVRUVLb9ixcvLte/+uoref09e/ao7+f7eJ8XfYwuDRs2lO9Pl1q1aqnKlCmjGj9+vHrbs77v+fPnqywtLVWnT59Wb+Pnbtq0abbn9fHxUU2ePFnreR0cHFRjxozR2vdFHx8UFKQqX768Kjw8XL3tm2++Ufn6+sr39jy6Xr9Jkyby+xAaGqretmTJEvkZ8HOzAwcOyO0NGzZoPXbfvn3qY9FnHwAwDWRmAcAgOJvG+CP93OBM2eeff67ucuDp6Uldu3aVDgBZcd3toUOHJNvGGUb+SJszlXv37qXmzZur97t//z5NmTJF/Zycpf3333+1sq4DBw5U3+bnUT665wVg/NH4xYsXJTPo4eGR47GPGTOGfvzxR8luHjlyRK/vNy+PyU12nOuSFZofpXNWl88dnws+d0lJSbJYizPhurLlWc/7hAkTtJ6XM/C6fkZ5fTyXAfCFz7lmtpYXE37//feUF5zN5jKTr776SivL//rrr8uiPP404IMPPpCSBqWuWVOzZs3U1/XZBwBMA8EsABiEUuMaHx+fq8dxuyr+yFmTj4+PfLTMH1lzjSjjEoYRI0ZIsMuLs/gxHKzyhQM0TRwsaX7syx8Dc0Cj4I+Gswazykf3EREREgRxTWefPn2eeexcf8klDm+88QatXLlS6myfJy+P0Rd3juCOCVlxoD527Fjy8/OTGk/NeuSs504Xru/N2qaMf0Zci6sPfR7P5RGM62Q18ZuV6tWrS51qbimPqVevXrb7goKC5E0Fv3lp27YtVaxYUcoj+MI/e75wKzmFPvsAgGlgaAIAGITSueD8+fO5elzWQFbJfnGAycGskmnlxVPdu3eX+k8OAhcsWCDZNV7sxftq4oBXEy8Ue+utt9QXDiR1dTPgy/r16yWDyTWaSj3qs3CWjwMwzjxyxlMfeXmMPrJ+3+zmzZs0cuRI+b452/znn3/KueN6WZb13OXmZ6TvsevzeOVnbWtrm21fXdv0wb8bTFe/YM2aaD6+U6dO0f/+9z85rpkzZ0rdN2ddlYysPvsAgGkgmAUAg+Cm/61bt5ZgkIPPnDzrvpxw1o4XdWXNlHLmTZ9grGfPnupglS9z5sx55v4caE6aNEm+F87qPgtnhnnhEy9e4kBHH3l5jPK43OIAjAPFrOcup64SphIQECBf+c1KVrx47EWekxe8ZcWdDXgRl5LB544LHPBzsM9dMHiSGJcozJo1S/0YffYBAONDMAsABqPUNg4YMIAeP36c7X7OqOalFyl/JJ01KOGsGwcRShmCofHH8ly7yTWoSoYvJ9wpgVtP8UAIfbOVeXkMZ165xvhFzx2/3vMCemPjmmfuwMAf/StZWsZ1tZxdzgvOmnJpBbdD4zdDmoE811Dz7ym/QeBWZlzXq4lrejl7y3XITJ99AMA0UDMLAAYtNdixY4d8jM71m6+99pq02+K6xE2bNsmCJ17slVv8HPyRP7e24hpPLy8vmdrF27Zs2ZIvP0HOwnF2lqdz/f777+oJYTnhwJpreXMzASu3j+GFccOHD5fWXnxOuD72eaUQXBvKk7m4ty23M+N6YV7gxm2muC9sQcEf3XOLNq5H5b6+r7zyitRNc6aWt+VlRC0/J7cp4++fxxPz7yO/GeChFfXr15fWWkpwywvz+M0F1/dytp8n2HFtMS9A03cfADANBLMAYFDckJ4/wt22bZt8BMs1tNzhYPDgwbRq1SqtleocZGhm4RQ8Bpc/ztXMui5dulQyuxwQc+cE/pieG/9fuHBBepE+7zlzwtm1rMMDFG+//bbUmXL3BM39OdunK5CfPXu2HA8HqFlfI7eP0WXo0KFyLrk/K/eN5eEDSjDLpRR8WxfuactDF3hhG2ciubSBOxhwFwgOdp81jjan5+XgkDsiaHrRx/OCKj4XS5YskUwonxMOODmwzTpEQxddr8+/GxwQc1CrDE3gUpMePXrI4jLGtdjc13b16tXy8+btw4YNk8V5yuvqsw8AmIYF9+cy0WsDAAA8F2f5edoWZ+MBALJCzSwAABQIPK43a/0wZ/N5Adjz2qQBQNGFMgMAACgQuLRg0KBBUiPLi8G4WwXXK/fq1UvqsAEAdEGZAQAAFKiAds2aNbJYjRe4cbs3XnQFAJATBLMAAAAAYLZQMwsAAAAAZgvBLAAAAACYrSK3AIx7EN69e1fmbOdlNCQAAAAA5C/uHBsXF0fe3t5kafns3GuRC2Y5kPX19TX1YQAAAADAc4SFhanHcuekyAWznJFVTk5O03IAAAAAwHRiY2Ml+ajEbc9S5IJZpbSAA1kEswAAAAAFlz4loVgABgAAAABmC8EsAAAAAJgtBLMAAAAAYLYQzAIAAACA2UIwCwAAAABmC8EsAAAAAJgtBLMAAAAAYLYQzAIAAACA2UIwCwAAAABmC8FsPktMSaM7j55QXGJKfr8UAAAAQJGDYDafvbnoGDWduZN2Bkfm90sBAAAAFDkIZvOZezE7+RoVl5TfLwUAAABQ5CCYzWcexWzla3R8cn6/FAAAAECRg2A2n3k8zcxGxyMzCwAAAGBoCGaNlJm9j2AWAAAAwOCsyUTS09Pp7Nmz2bb7+fmRm5vbcx977do1sre3J19fXzKPzCzKDAAAAAAKTTCbkJBAderUoQoVKpCjo6N6+5dffkndunXL8XGHDh2ifv36yeP5UrNmTfrrr7/I29ubCiKUGQAAAAAU4jKDxYsX0+nTp9WXZwWy8fHx9Morr8g+ERERFBUVRRYWFjRo0CAqqNzVZQbJpFKpTH04AAAAAIWKyYPZBw8eUEhICCUlPX+B1Lp16yg6OpomT54sQayDgwNNnDiRtm/fTtevX6eCnJlNTkun2MRUUx8OAAAAQKFi8mC2b9++1LFjR3J2dqa33nqL4uLictz3xIkTFBAQQO7u7uptjRs3lq8nT56kgsjexoqc7TKqOdDRAAAAAKCQBLNWVlY0b948evTokWRVucRg69at9N577+X4mPv372sFsqxEiRJkaWkpGVtdOOMbGxurdTE2D+eni8AwOAEAAACgcASzXCIwcuRICWpZ1apVafz48bR8+XJKS0vT+RgbG5ts5QgpKSnS3YDv02XGjBlUvHhx9cUU3Q/cnZ7WzT5GRwMAAACAQlVmoKlMmTKUmJgoC7t04UA0PDxca9vdu3fV9+kyYcIEiomJUV/CwsLI2NDRAAAAAKCQBbO6Fnzt2rVLesx6enrKbc7QcvkBLxJjbdq0kWCWtyk2bNggWV6ldjYrOzs7cnFx0boYm4fz05G2KDMAAAAAKBx9Zrle9sKFC9Jmy9XVlTZt2iTb5syZoy494MVg3It24cKFNHjwYGrWrBl16tSJBgwYQLNmzZIgd9KkSVKeUKxYMSqo3J2e1syizAAAAACgcASzY8eOpRUrVtCiRYukrKB8+fK0Z88eatKkiXofDmpr1aqlNRFs1apVNHPmTJo+fbpkXTmoffvtt6kgwwIwAAAAgPxhoSpinfy5mwEvBOP6WWOVHGw5H04jlp6kun6utPqdpkZ5TQAAAICiEK8VqAVghZX708EJ0fHoZgAAAABgSAhmjdjN4H7886ecAQAAAID+EMwagUexjG4Gj5PT6Emy7h66AAAAAJB7CGaNoJidNdlZZ5xqjLQFAAAAMBwEs0ZgYWGhLjWIQqkBAAAAgMEgmDUSt6cjbR+i1ywAAACAwSCYNZLiDjbyNS4x1VgvCQAAAFDoIZg1EheHjPkUsYkpxnpJAAAAgEIPwayRONtlZGZjnyCYBQAAADAUBLNGz8yizAAAAADAUBDMGomLPTKzAAAAAIaGYNZIXJ4uAEPNLAAAAIDhIJg1dpnBE5QZAAAAABgKglkjlxnEoZsBAAAAgMEgmDV6mQEyswAAAACGgmDWSLAADAAAAMDwEMyaYGiCSqUy1ssCAAAAFGoIZo2cmU1JU1FiSrqxXhYAAACgUEMwaySOtlZkZWkh19GeCwAAAMAwEMwaiYWFBbnYK+25MNIWAAAAwBAQzBqRszIFDO25AAAAAAwCwayRSw1YQnKaMV8WAAAAoNBCMGtETnYZZQaPkxDMAgAAABgCglmTZGYxOAEAAADAEBDMGpGT7dPMLMoMAAAAAAwCwawROdo9zcwmITMLAAAAYAgIZo3ICZlZAAAAAINCMGtEyMwCAAAAGFZGEaeJJSUlUe/evSk8PJw2bdpEXl5eOvd78uQJNW/ePNv2KVOmUJcuXaigc0JmFgAAAKDwBbMffvghhYWF0ZkzZyg5OTnH/dLS0ujEiRO0cOFCql69unq7v78/mQN0MwAAAAAoZMHs2rVradeuXTRz5kzq1q2bXo+pXLkyBQUFkblBn1kAAACAQhTMcjZ2xIgRtHnzZoqOjtb7cR999BFZWFhQQEAAvf3229SkSRMyB8jMAgAAABSSBWBcMjBgwAApMahdu7bej2vYsKEEwFwn6+HhQS1btqTly5c/sx43NjZW62IqTqiZBQAAACgcmdkvvviCbGxsJJjVl6OjI+3fv5+srTMOu23bthKs8nP069dP52NmzJghr1UQoJsBAAAAQCHJzC5evJhu3rxJ9evXl/rXUaNGyfbOnTvTt99+q/MxlpaW6kBWwQEtd0G4f/++zsdMmDCBYmJi1BcubTAVp6eZ2QRMAAMAAAAw78zs+vXrJauqOHr0qAS0nEmtUaOG3s9z9+5dsrKyInt7e53329nZyaUgcHo6AexxMiaAAQAAAJh1MJs1YH306JF8rVmzJvn4+Mj1+Ph4atWqlbqP7Jo1a6hUqVLUqFEjuT8kJES6IHTv3p2cnJyooHNUMrNJaaY+FAAAAIBCoUBPAEtNTZW+skqng0qVKtGkSZOoZMmSVLFiRQl827VrRwsWLCBz4PQ0mE1OS6fk1HRTHw4AAACA2TN5n1nNLgXHjh2TQFXh7Ows25ShCFWqVKEdO3bQgwcPKCoqisqVK1dgSgj04WCbUWbAniSnka11gX4vAQAAAFDgFZhglgPXrIMQuBZW13AENzc3uZgbDl5trSwlM8t1s8UdbUx9SAAAAABmDalBE2VnE7AIDAAAAOCFIZg1MrunpQVJqJkFAAAAeGEIZo3MzgbBLAAAAIChIJg1MjvrjDKDpBR0MwAAAAB4UQhmTVZmgF6zAAAAAC8KwayRoWYWAAAAwHAQzJqqzAALwAAAAABeGIJZUy0AS0GZAQAAAMCLQjBrZCgzAAAAADAcBLNGhjIDAAAAAMNBMGtk6GYAAAAAYDgIZk1WM4s+swAAAAAvCsGskaHMAAAAAMBwEMwaGcoMAAAAAAwHwayRITMLAAAAYDgIZo0MNbMAAAAAhoNg1kRlBslpWAAGAAAA8KIQzJqqzAATwAAAAABeGIJZI8MEMAAAAADDQTBrqprZ1DRjvzQAAABAoYNg1sjQzQAAAADAcBDMmqrMABPAAAAAAF4Yglkjw9AEAAAAAMNBMGtkdjZPuxmkojUXAAAAwItCMGtk6GYAAAAAYDgIZo3MVl0zi24GAAAAAC8KwayRITMLAAAAYDjWVEBs2LCB7t27R/379ydHR8dn7nv79m3at28f2dvbU5s2bah48eJkjq25VCoVWVhYmPqQAAAAAMxWgcjMbtmyhfr27UvDhg2jBw8ePHPfxYsXU6VKlej333+nmTNnUmBgIB0/fpzMbWgCS07DIjAAAAAAsw5mORvLQeyXX36p174jRoygr776SgLgI0eOULt27Wjw4MFkbmUGDB0NAAAAAMw4mE1PT6fXX3+dxo4dS7Vq1Xru/mvXrpWvb731lnrb6NGj6cKFC3T27FkyB7ZWGsEsBicAAAAAmG8wO2PGDPn6wQcf6LX/+fPnyd/fnxwcHNTbqlWrpr5Pl6SkJIqNjdW6mBLXyGJwAgAAAICZB7MHDx6kOXPmSO2rvougOBAtUaKE1jZe/GVlZZVjkMoBM++jXHx9fcnU0NEAAAAAwMyD2XfeeYeaN29Omzdvpvnz58tXtnz5cjp8+LDOx3BGNi4uTmvbkydPKC0tTStbq2nChAkUExOjvoSFhVGBmQKGMgMAAAAA82zN1bFjR4qOjlYHrnfv3pWvp06dIj8/P2rUqFG2x3DngpUrV0qtraVlRhweGhqqvk8XOzs7uRQkKDMAAAAAMPNglttqadq+fbtkZ2fNmkU+Pj6yLTk5WVpxtWzZkipUqECdO3emjz/+WDoZvPzyy7LPsmXLyMvLixo0aEDmAmUGAAAAACYKZvfu3UupqanUunXrfG/4n5CQIG27Fi5cKMFslSpV6L333qOBAwfSmDFjpCftvHnzaMmSJWRjY0PmODgBAAAAAIxYM3vz5k3q0KEDBQQE0NSpUw1Wg1qmTBlpueXk5KTexuUBvK1ixYrqbd999x399ttvEsjy/byQ7LXXXiNzogxOSEpJM/WhAAAAAJg1CxXPVM2l8PBw6ULAGdOrV6/K4II333yTevToUeDqU7Pirgfc1YAXg7m4uJjkGF775RAdDn1A/+1Xh7rW8jbJMQAAAAAUhngtT90MSpcuTePHj6eQkBDas2eP1LgOHTqUvL295eP/nHq+QgaUGQAAAAAUkNZcPLmrcePGUtP66NEj2rFjB9WsWZNeffVVqXmF7NDNAAAAAMCEwSxXJnBGdtCgQVSqVCmaNm0ade/enW7cuCGjZc+dOydZ2wULFhjoMAsX9JkFAAAAMFE3A868Dh8+nG7dukXdunWjv//+WxaEKX1flRGz/fv3l30g58xschq6GQAAAAAYNZjlLgIczA4ePJg8PT1z3I/34clc8IwyA0wAAwAAADBuMNunTx+99nN1dc3L8RSxBWAI9gEAAACMXjObmJhI33//PbVv314GGbRp04amT59O8fHxL3QwRYW6zyyGJgAAAAAYNzPLI2ZbtGghwxO4YwH3mI2IiKCffvpJJnEdPXqUnJ2dX+yoCjl0MwAAAAAwUTC7du1aioqKokuXLpGbm5t6O3c0aNKkCS1evJhGjRploMMr5GUGqJkFAAAAMG6ZAS8A4/ICzUCWOTo6UteuXen+/fsvdkRFKjOLbgYAAAAARg1ma9euTUeOHJFyg6y9Zw8cOCBDFEDfmlksAAMAAAAwaplByZIlydrampo1a0ZDhgyREbZcdvDHH39IX1lbW1vasmWL7Ovv70+VKlV6oQMsjDDOFgAAAMBEwezq1avpzJkzcv3kyZPZ7udSA8WHH35IX3311YseY6Fjiz6zAAAAAKYJZj/44AO5QN6hmwEAAACACfvMwovBAjAAAAAAE2VmFTdu3KCtW7fS7du3qXTp0tS6dWuqXLmygQ6rcEPNLAAAAIAJM7PffvutLOyaNGkSrV+/nqZOnUrVq1eniRMnGuiwCjd0MwAAAAAwUTDLwxI4aF24cCFFRkbSqVOn6O7duxLUzpkzh/bu3WugQysCZQYYmgAAAABg3GB2z5499Morr1D//v3JwsJCvb1Tp040bNgw2rlz54sdURGAMgMAAAAAEwWzHMCmpelu9p+amkqWllhT9jzoZgAAAABgGLmOPNu2bUvr1q2j+fPnS/CqTP9auXKlbOvQoYOBDq0o1Mymy7kDAAAAACMFs4GBgTR37lx6//33ycXFRaZ8OTs70xtvvEFTpkyhRo0a5fFQil6ZAcexyWnppj4cAAAAgKLVmmvo0KHUrVs32rVrl7o1V/PmzcnX19fwR1gI2T/NzLLElHR1cAsAAAAA+RzMLlmyhKKjoyUz27dv39w+HHicrZUlWVlaUFq6ip4kp1FxBxucFwAAAABjlBkkJSVJey7IO15E52iTkY1NSM6oOwYAAAAAIwSz7du3l/KCiIiIPLwcKBxslWBWd2cIAAAAAMiHMoMrV66QjY0NVaxYkdq1a0eenp7Zgt1evXrl9mmLHMenweyTFASzAAAAAEYLZp88eUJVq1aVC+P6WU2PHz/O88EUJQ62Gaeea2YBAAAAwEjBbFBQENWqVYv8/Pyy3RceHk4pKSl6PU96ejr9+eefMgL33Llz5OrqSp07d6bp06eTu7u7zsfEx8dLG7CseLTu4MGDyRwzsygzAAAAADBizewff/whAWhu78vqwoULUnvLPWujoqJo8+bNdOTIERo4cOBzH3vo0CEZNqBczC2Q1S4zwAIwAAAAgLwy6OzZmJgYnZlTXWrUqEG//PIL1atXjxwcHOQ2D17gQLUosFd3M0CZAQAAAEC+lxls2bKFVqxYIRlVrpvVVSv777//yljb3OKSg8uXL9OyZcuoT58+z92fyxESEhIoICCAhg8fTqNGjSJLS4PG5cbLzCKYBQAAAMgzy9wEnKmpqfJVua554a4Gv/76K3Xs2DFXB8AdEaysrKhKlSrk4eFBP/zwwzP7s7799tt04MABunfvHn366ac0ceJEmjZt2jP74sbGxmpdCgLUzAIAAAC8OAsVF53mAte5xsXFyThbQ+FFY+fPn6dhw4ZRiRIlaNu2bXo/lgPZWbNm5RikTpkyhb744gudJREuLi5kKl+uv0i/HbhOI1oG0PhOlU12HAAAAAAFDcd1xYsX1ytey/Vn861btzZoIMu4b22dOnVo9uzZtH37dik50BfX2nJwndMQhwkTJsiJUC5hYWFUkDKziegzCwAAAGC81lzs7Nmz9OOPP9L169cpOTlZ676+ffvSyJEj83QwSlsvLifQF7f14gVknNHVxc7OTi4FdwIYuhkAAAAAGC2YvXXrFjVt2pQaNWpEdevWlayqJn9/f72eh1tyca0sL+biWtnTp0/TBx98IM9doUIF2efRo0cSpCp9ZP/3v/9RWlqaZIa5L+2mTZvoq6++onfeeYdsbW3JnKBmFgAAAMAEwSyXATRp0kQ6F7yIAQMG0IwZM6RsgRdz+fj4UM+ePemTTz7J8TH9+vWTx7Rq1Up605YvX16CWe5oYG7QzQAAAADABMGsvb09lStX7oVfmDOuvHCLLznh7Kvm+jS+zcErX8wd+swCAAAAvLhcLwBr0aIF7d27lx48eGCAly+6HG0z3kckYAEYAAAAgPEys8HBwbJAi/vCco/YrBO/2rdvT7169cr7ERURmWUGWAAGAAAAYLRglqd/Va1aVT2QgC9ZJ4GB/t0MniAzCwAAAGC8YLZr165ygReDBWAAAAAAJqiZBcNwtHlaM5uchlMKAAAAYMxg9vbt2/T6669La6zPPvtMPbzg66+/zutxFDluxWzVwWxsYsawCAAAAADI52A2MTGR2rRpI8ML6tevLzW0rFq1arR06VIKCQnJ7VMWScXsrKmkS8ZkstAo1BkDAAAAGCWY3bZtG5UsWZKWL19ODRo0yHwiS0sZZrB27do8HUhRVN6jmHy9Fhlv6kMBAAAAKBrB7M2bN6lmzZpynVt0aSpWrBjFxMQY7ugKufKeTvI1NBrBLAAAAIBRglk/Pz86depUtmA2JSWFNmzYQJUrV87TgRRF5T0zMrMoMwAAAAAwUjDbsWNHmf41YsQIunr1KkVFRdE///xDbdu2pYcPH2JgQi4EPM3MXotCZhYAAADAKMGsra2t1M2GhYXRTz/9RIsXL6aePXtSeno6bd++nRwdHfN0IEVRwNPM7I37CZSWrjL14QAAAAAU/qEJzNfXlzZu3EixsbEUHh5O7u7u5OHhYfijK+S8XR3I1tqSklPT6c7DJ+TnjjcCAAAAAEYbmuDi4kKVKlVCIJtHVpYWVN4DpQYAAAAAeYUJYAWkowHqZgEAAAByD8FsAek1GxqNwQkAAAAAuYVg1sQCvJ5mZjE4AQAAAMA4wSy35OLhCYzH2s6ePZuGDh1KBw4cyMvTFWlVSxeXr8dvPqTLEXGmPhwAAACAwh3McgeDHj16SIsuxu25vvrqK+k32759e7pz505+HGehVamUM3WoWlJac339b4ipDwcAAACgcAez3GO2evXqVLp0abm9bNkymjt3Lq1du5ZeffVVWr16dX4cZ6E2qnWgfD124wGpVOg3CwAAAJBvwWxkZCS5urrK9fj4eDp58qRMBWMBAQEUHR2d26cs8jg7y226HiWkUERsUpE/HwAAAAD5FsxWq1aNNmzYQGfOnKFvvvmG6tWrR8WLZ9R9Xrx4kapWrZrbpyzy7G2s1P1mL92LLfLnAwAAACDfJoC1aNGCWrVqRbVr15ahCevWrZPtt27dolOnTtGiRYty+5RARFVKu9CVyHgKDo+j1pW8cE4AAAAA8qubwdKlS+nRo0ey6Ktly5ayzdnZmXbu3El2dnZ5ecoir3JpZzkHF8ORmQUAAADI99ZcHMxyRwOlNde4cePo+vXreXk6IKK6fiXkPOwKjqT4pFScEwAAAAA9oDVXAdHQ301G23Ig+8/J26Y+HAAAAACzgNZcBYSFhQUNbFRWrn+77TJNWXeBwh4kmPqwAAAAAAo0tOYqQLrXLiMtuh4mpNCigzckqAUAAAAAA3Yz4NZc//nPf6Q115o1a7K15uLpYPpKSEigVatW0blz56R3befOnalOnTrPfdymTZtox44dZG9vT71799brMebAzcmW6vi6ymhbdvT6A1MfEgAAAEDhysxqtub69ttvJbDVbM2lbzDL+9etW5f2799PZcqUofDwcGrcuLE857PwQrPXX3+dHBwcZBFaw4YNJSAuLCZ2rqK+HpuYgolgAAAAAM9gocrj/NSYmBgJKLmjAXv48CE9efKEvL299Xo8B6Lp6enk5uam3vb555/Tr7/+KoGtLiEhIVSlShXpbdulSxfZNn78eOlte/v2bbK2fn6iOTY2VjLJfPzcJ7cgSk5Np6qfb6HUdBUdmtCGShd3MPUhAQAAABhNbuK1PLXmYvwCSiDLSpQooXcgy7isQDOQZQ8ePCAvr5wHBvDkMX5cp06d1Ns4SxsREUFHjx6lwsLW2pL8n04EC7kXZ+rDAQAAACg8NbMsNTVVMqjr16+XjGjp0qWpXbt2NHr0aKljzY0ffviBLly4QJcvX5ZM7YoVK3Lc98qVK+Tr60tWVlbqbf7+/uret02aNMn2mKSkJLloRvrmoGIpZ5kItu1iBLXCRDAAAAAAw2RmuSqBM6MTJkyQTGyvXr2ofPny9PXXX0swmZycnKvnCwgIoBo1alClSpUkqD106FCO+3IZA08a0+Tk5CTBLS8m02XGjBmSRVYuHAybg261MrLcfxy5RZvO6S67AAAAACjqcp2Z3b59u3QtuHTpkmRkFTNnzqRGjRrRn3/+SQMHDtT7+ZTaV8ZdCd555x1ZRJa1BIFxzQTX2mriTCtPIcupnoKD7g8++EBrf3MIaF+qVoreblGeftkbKm26Xq6Rea4BAAAAII+ZWQ5iX375Za1AlnHWk9tkBQcHU15xmy8uCQgLC8uxLRiPzE1MTFRv48BauU8XOzs7CXQ1L+ZiSNNyZGmR0aKryYwdFB2fWS4BAAAAAHkIZj09PaUvrK4mCKdPn5b79XH48GFZ8KVp+fLlEhRXqFBBXVYwYsQIdelB9+7dpa524cKF6sfMnTtXOhzUqlWr0P08uYtB+6ol5frdmETacSnC1IcEAAAAYN5lBjzY4P3335fAkgNN7hHL3QS4PdaePXvop59+0ut54uLiqFmzZlSuXDny8PCQIQxRUVG0dOlScnR0lH04S/vzzz9L+QL3oOVs8I8//kjvvvuuDE7gYJgzwXy9sJrVuxadv7OP7jx6QjfuY7wtAAAAwAv3meUAcsyYMVI/qzycF39xZ4KgoCC9n4cXbR04cIDu3btHPj4+8hxcFqDgxWS//fYbtW7dWhaIKW7cuEH79u2Tfdu3by9twfRlDn1ms1qw/zpN3XCRXq5RiuYNqGfqwwEAAADIV7mJ13IdzCoLrjiA5NrVO3fuSMZUyaYWdOYYzHJ5wVu/H6eqpV1o03vNTX04AAAAAOY7NGHBggXqEbbcU5Zba5lLIGuuyrpnDFC4ef8xxtsCAAAAvEgwy71lc+o2APnD182BLCyIHienUXR87vr4AgAAABRmuQ5mu3btKu2weFIXlxtA/rOztiLv4g5yfVdwJE45AAAAQF6D2d9//10WYPXr148cHByoVKlSWpcvv/wyt08JeuhRJ2Mi2KdrztO83Vfpz2O3cN4AAACgyMt1a65WrVrR//73vxzvr1q1apE/qfnhg/aVZHjCsRsPadaWENnWrIInlXHNyNgCAAAAFEW5DmZ5QAFfwLisLC2oV10fCWYVF+7EIJgFAACAIi3XZQaM23Fxb1hN9+/fp5s3bxrquECHDtVKad2+GB6L8wQAAABFmmVe+n517NhRBhpoPZGlJXXp0iVbkAuG4+ZkS9NfqUE2VhZy+/vtV2jo78dQPwsAAABFVq6DWR4dW6NGDfLz89PazkMU2rVrR3/99Zchjw+y6N/Qj34f0kB9e/ulSPp87QVKTEFnCQAAACh6ch3MPnz4kCy46akOvD0yEq2j8ltVbxey1PgRJKWm08mbmbW0AAAAAEVFroPZhg0b0oYNG+jatWta27m8YPny5dSgQWbWEPKHq6Mt/TwwiH4eWI9eqVNGtu2/Go3TDQAAAEVOrrsZ1K1bVwYn1KpVi/r06UNly5alu3fv0qpVqygoKIg6d+6cP0cKWtpXLSlfY5+k0D+n7tC83deocYA7Na/giTMFAAAARUaeuhksXryYZs2aJcMT/vzzT7p06RJNnDiRNm7cmGMJAuRfh4PSxe3l+uR1F3CaAQAAoEixUKlUKipCuBtD8eLFKSYmhlxcXKgwiIxNpAbTd8j1brW86eUapaljde02XgAAAACFMV7LU2YWChYvF3sq6+4o19eduUsjlp6gBfuvm/qwAAAAAPIdgtlCopq39ruWr/8NpjuPnpjseAAAAACMAcFsIVHO3Ul9va6fKyWmpNMHf56mlLR0kx4XAAAAQH5CMFtIDG1enlpV8qT/9qtDs3rXIidbKzpy/QEtO3LL1IcGAAAAkG8QzBaiUbeLhjSgrrW8KdCrGI1uW0G27wrBEAsAAAAovBDMFlLNK3jI190hUdTgP9vpckScqQ8JAAAAwOAQzBZSVUplLgiLjEuin/eEmvR4AAAAAPIDgtlCytLSgjpWy+w1u/dKFKWnF6mWwgAAAFAEIJgtxCZ1rkIfdahIlhZEUXFJdOLWQ/V9qWnpVMTmZQAAAEAhhGC2EPN1c6R321SgHnXKyO1N58Lla2JKGnX6YR91nrMf2VoAAAAwa9amPgDIf52ql6bVJ+/QwgM36ElyGjUJ9KArkfFy3wcrT8vtV4N88aMAAAAAs2OhKmKfNedm1m9hwZnYelO30ePktBz3OTelAznb2xj1uAAAAABeNF5DmUERYG9jRRNerkLF7HJOxNeYspWmrLtAw5ccp4NXo416fAAAAABmm5lNSEigmzdvko+PDzk7Oz9z3/T0dDp58mS27f7+/uTu7q7X6xXFzKymvZej6L87r9CxG5mLwbJytLWii192NOpxAQAAAJhVZvbKlSvUu3dvKlWqFPXs2ZNKlixJ/fr1o8ePHz8z8K1fvz4NHDiQRowYob4cOnTIqMduzlpU9KRVI5pIl4OcJCSn0Z7LUdT2m910SqMDAgAAAEBBY7Jg9ty5cxK8Pnr0iC5dukSXL1+WoHTcuHHPfezChQvp+PHj6kuXLl2McsyFiU8JR/X1TztXoW61vLXuH/TbUboW9Zj6/3rEBEcHAAAAUMC7GXA2VhOXGXCmdvPmzc99bGRkJF24cEHKCxwdM4My0F9JF3v19S41val3PR+ysrSgf07d0drvSUoaJaWmkZ21FU4vAAAAFDgFagHY6dOnqVy5cs/dj8sMevToQa6urjRo0CCpq4DcqVQqsz65pIsduTra0nd9a1OTgOy1xyuOhqEfLQAAABRIBSaY/e2332jXrl00fvz4HPexsrKiX375hR4+fCg1t2fPnpXHjBkzJsfHJCUlSbCreQEiNydb2v5BCzo4vg1ZWFioT0kFr2LZTs/kdReo8cwddOjafZw6AAAAKFAKRDC7fv16GjlyJM2dO5eaN2+e434ODg40bNgwsrTMOOzKlStL8LtixQpKS9PdQ3XGjBmyGk65+PpiOIAi0MuZvF0dtM5Xswqe6usN/N1oVOsAsrexpIjYJKmjvRGte4FeerqKdgZHUHR8Uq5//gAAAABmG8xu3LiR+vTpQ7Nnz5aANre8vb0l+xoVFaXz/gkTJkhbB+USFhZmgKMuvNpXLUnvta1AlhZEfer50LiXKtPJz9pTXT9XSk5Lpzk7rtCGs3cpIjZR63Frz9yhNxcdpzcWHDXZsQMAAEDRY9JxtrzYq1evXjRr1iwaPXp0tvs523rq1Cl1H9nExESyt89cuMR27Ngh93l5eel8DTs7O7mA/t5vX5FGtAwgB9uMRV+OttbUJ8iXTt56RKtP3ZELK+fuSK0qedGUbtXo7xMZ2y6Go4wDAAAAikAwu3v3bnrllVekPVeTJk2kxZYckLU11a5dW67HxcVJX1luxTV48GD66aefZJFYt27dZPHXpk2bZNuPP/6oLj0Aw1ACWUXbytnfLNy4n0CLDt6gyxFxdFCjnjbmSQoVd8BoXAAAACjEwez58+epevXq0m+WBx8oeAoYL+qSg7O2pnr16pGHh4fcHjt2LK1atUpqZLmsoHz58nTgwAFq0KCBqb6NIsPLxZ5eqlaSTtx8SN/3rUNj/zxF0fHJcp9mIMuuRsZRvbJuJjpSAAAAKEpMPs7W2Ir6ONsXwb8qqekqsrGypNjEFPpw5RnadjEi234zetagfg38KC4xhWZsDqbTtx7RjwPqkr+Hk0mOGwAAAApvvGbSmlkwL9zCy8Yqo42Xi70NNfR30xnMLjxwnRbsv06Pk1IpPCZjodjPe67RzF41jX7MAAAAULih0BTyrGpp3e+ULkfE09XIeHUgy9afuSvBLQAAAIAhITMLedY4wJ3GvVRJyge4VVddvxI0cMERik3UDlq56wEvFtsRHEndannrfK7TYY/Iz81RhjkAAAAA6AvBLLxQ2cGo1oFa2xYMrk8f/3VWhifEJabSRx0q0qOEFJq//zodvBpNDjZWtO3iPfIt4UjvtgmU51hx9BaNX31ORukuG9YIPxEAAADQGxaAQb5ITEmjvZejqE1lL9p3JZqGLDqWbZ81o5pSKRd7ajRjh3rb0YltpXMCAAAAFF2xuVgAhppZyBf2NlbUoVopsraylLG4uvyw/bJME9O07VL2BWUAAAAAOUEwC/nOyc5aetTaWlvSgkFBtPjNjL7Au0KiaNrGS3JdGbKw81Kk+nHp6Sp6+Dijly0AAACALghmwSjmDahHxya1o7ZVSlLTQA9qXclT6/6PO1aSr2fvxNC+K1H0JDmN5uy8QnWmbqNf94bSkkM3KCk1DT8tAAAA0IKaWTCZXSGRNGThMSrr7kgbxzSnGlP+JWWEx6DGZen3Qze19m9XxYt+GRhElpYW6iEO687cpbLuTlTb19UU3wIAAADkA9TMglloXcmL1o5qSkvfakjF7KyprJuj+r6sgSzbfimSDlyLVt9ecSyM3ltxmt5adIzS0ovUIDsAAAB4CmUGYFK1fF3J92kQ6+ls99z9D4fel69XIuJo2oaLcv3+42TpUwsAAABFD4JZKDC4XEAXK0sL6lmnjFw/dO2+DGZo/91eepycWUO7OyRz4ZhSwtDhuz107nZMPh81AAAAmBKGJkCB8UnHyhSfmEpR8Ul04uZD8nVzoL9HNiFbK0t6mJBCq0/doZO3MjOwNcoUp+61vaUjwsrjYdS3vi+VdLGnncGRNHzJCdmn69z9NLpNoAx34HZhAAAAULggmIUCg8sMfhpYj+KTUmnrhXvUtnJJKu5oo27dVbq4PYXHJMptnizGASpnZ/84couuRz+mMctPUaVSLrT86C2t5/3vzqvy+KHNy9ON6Mfy/NXLFDfJ9wgAAACGhTIDKHB4MVjPuj7qQJbx2FvuZNC7ng+93siPhrcMkG2879KhDaWHLWdtswayilO3Hkm7rw7f7aXuPx6gO4+ePHeCGY/f5V63AAAAUHAhMwtmo4ZPcZrdp1a27WVcHejVIB9aelh3IMuuRsbT5vPhlJyWLre/3XqZYp4kU8WSzvRxx8rZ9v9i/UUJjD/tXEUyugAAAFAwoc8sFArR8Uk0dcNFql/OjZoFetCPu65S99plaPulCFp08AZxa9rKpVzoYnhstseufqcJ1fUrobWt3PiN8tXCguj6jM5G+z4AAACActVnFsEsFHoNp2+niNikZ+7zQfuK1LlmaVp88Aa1ruxFgxcek+1cvhAytaOUNPxz6jYF34ujT16qrB7cAAAAAIaHoQkAGrrW9FZf55pbrrNlvwysRw5POxx8u+0ytf1mjwxrUAJZlpyaTrsvR9HCA9fp/T/P0M97QiXbCwAAAAUDMrNQ6KWmpdOna87LIrDFbzWQkoSUNJWMwL0fn0TfbLtMy47kXG+b1YftK9LothXUt3mRmJKpvRoZRz4lHNEGDAAAwEiZWSwAg0LP2sqSZvaqqb7NvWgV7sXsaFr36tSqoqdc544Hn/x9lhr6u5GDrZW0/crqzO3MXrfcEuyVeQeoTSUvalbBgz5YeYZ8SjjQoiH1KdDL2QjfHQAAQNGGzCxADjiDGzRtu877uL7Wt4QjRcQm0j+n7sgCMzcnW4qOT5b7u9bypv/2q4NzCwAAkAfIzAIYgEcxO/rhtdr057EwOnjtvtZ9G8+Gq7sdMG5HqwSybMPZu1SjjAv1rucrQe6xGw/o9K1HNLhpOelhywvL7KwxkQwAAOBFITMLoIcLd2MoODyOrK0saMPZcNp2MXMRmKOtFSUkp8n1fg38ZLRu2tNhCwGeTrRyeGPqPGc/3YvNmF6m9Mbd8WFLCYZvRCfQwAVHqE+QD417KXvPWwAAgKImFq25DHNyAHKy5fw9GrH0hFxfNrQh7QqJpECvYtSnni+Vn7hJa9+mge504Kp2ZpfN7FmDvtoSTA8TUtTbgqd2zHHxGI/hTUhKJS+Nml8AAIDCCMGsgU4OQE64ZdeHq85Q6eL2NPHlKlr3zdh8SVp4Odla0eOnGVsFTyrjcoSdwZE6n5cD4u/71pbhDtxSjDPBXJfLmdxucw9Q8L1YGevLvXABAAAKKwSzBjo5AHnBGdTN58JlkdjQ34+r620/6lCR3m1TgQ6H3qfXfjn83Od5pU4ZSklLp43nwunL7tXpszXnZbu1pQUd/7QdFXewofn7rksA/KzglgNvPiau3QUAADAHCGYNdHIADNERYczyU3Q67BGtH92MAjyLST1tu2/3SFsvtmpEY/IsZketZu/W+3mHtyhPTnbWMuyBrR3VlCqVcqbdIVHUOMBdAl3FBytPS53v6pFNqHqZ4rLtUngs2VhZSiAMAABQ0JhNMLt9+3b6448/KDQ0lHx9fWnYsGHUsmXLZz7myZMn9PXXX9OOHTvI3t6e+vbtS2+++aber4lgFkyBs6PcwUDBNbZDFh6ToJOzrBxYcpAbkKXeNi9eqlaSfh4YpL5dbvxG+VrN24X+GNqQktPSqcF/dsi2kGkd0VUBAAAKHLNozTV37lxau3YtDRgwgAYPHkz79u2jtm3b0tKlS+m1117L8XGvvvoqXb16lWbMmEEPHz6k9957j8LDw2nSpElGPX6A3NAMZFnrSl60cEh9cneylUCWWT2dIqaJM6fXouKJ33La21jSpM5V1eUGOfn3QoS0BouOS6JjNx+qt1+4G0sNpu+g/g381NvO34mhemXd8MMEAACzZbLMbEJCAjk6OmptGzRoEF2+fJkOHTqk8zEHDx6kpk2b0okTJ6hu3bqybc6cOTRhwgSKjIwkJyen574uMrNQkK0+eVumiL3TKoBq+hSnlhW9aP/VaNp28R6NbVdROh3UnbpNvX+NMsVpZq8atOTQTVpxLCzXrzehU2XqUsubSjjakKMtBgICAEDBYDZlBlmNHDmSjh07RsePH9d5/9SpU2nevHmSiVVcv36dypcvT9u2baN27do99zUQzEJBxv873nqQINPFLHVkatn6M3cpKTWdOlYvRTZWFlImkJqWTuvP3iU3Jzuat+sqxSWmSkcEBWeAhzQtR4sO3tAa7sC41y3fP7tPLapa2kWGRSivPX9fKC09fJPm9q+rrrfNeryLD92k2r6uVMvX1eDnAwAAiqZYcygzyIqDUq6fHT9+fI773Lp1i7y9vbW2lSlTRn2fLklJSXLRPDkABZWFhQWVdX/2Jww8KjcraytLeqWOj1xvWdFTvnJLr4bTd6i3cSeF7rXL0JBFx+hqZLz6sfx2lgPcwQuPyW0Oeid3rSY1vNM2XpJtPecdpBOftSNn+8yFZWzdmbs0ed0FuX5jZme6Ef2YypRwUJdOAAAA5LcC8S/OgwcPqFu3blI6MG7cuBz3S0lJITs7O61tNjY2ZGlpKffpwrW1HNkrF15oBlAUlNQYrhBYMqNrga+bI20d24JCp79Mw1uWp07VS9GsXjWlzECx8MAN6b5w9PoD9TZeNDZ8yQlKSk2j9HQVjV1xiib+c072U0xYfVY6Mvxn4yUZKsFje/UVGZdIby06Rlsv3JPbF+/G0kerzkg3CAAAgAJdZvDo0SMpD7C1taV///2XnJ2dc9z3ww8/pA0bNlBISIhWIOzu7k6rVq2i3r1765WZ5YAWrbmgKDgT9og2n79H77ev8MyuBX8cuUmT/slcWMaLzdLTM4LYyqWcKexBggyA6FyjNI1sFUBd/rtf9ivlYq81plcTZ4MXDq5PUfFJdCUinppV8JDtUXFJUp6gOclMGTTB9oxrRd1/PECPElKoob8b/Tm8sWyfuuGiBLm8cC6nKWkAAFA4mE2ZAR9ghw4dJLu6ZcuWZwayjDO3vOCLA1g3t4wV2EeOHFHfpwtncrNmcwGKilp61rJyhwOu0y3n7kSf/H2WDoVmDHqws7akWb1rUuyTVBqy6KgMcLhwN0b9uJwCWbbnchR9/PdZ2n8lWvbjutwaPsXp5M2HEiT/2L+u1NpyF4cDV6PVj+vz0yEJZNmRp9lhLnlYsP+6XOesb486GeVFAAAAJsvMcsTdvn17srKykkBWV9QdHx9PrVq1oilTplCXLl0oLi6OAgMDqX///vTdd99JxpWfw9ramnbu3Kn362JoAkDOuIzg8PX7dOt+gkwWU8oVVhy9ReNXn8vXU+fqaKMOZBXXpr9M4TFPqNlXu+T22HYVpLODppgnKRT7JIWmb7pEQ5uXp3plS+TrcQIAQP7KTbxmsprZ2bNn09GjR6VXbJs2bSgoKEgurVu3Vu+TmpoqbbiiozOyNpy5/fvvv+mvv/6ShWBeXl4yRGHx4sWm+jYACh3uZNAkwINea+CnVXfbt74vNX9aKsAcbTM+6m/g7yatxP4e2Zic7TM/7Pmuby3ycraTbgn64AzyN31qZdvOpQUcWCvm7b4m2V6u1+UFZ/uuRFH9adup+axdUlIx+Lej6n15P67ljU3UXVOv4LHB3LlB83UAAMA8mCwze+fOHa0WWwrOstauXVuup6Wl0alTp8jf31/qYhW8nfvRcvkAt+XKDWRmAfKOywEGzM8o7dk7rjU52FqRk52VukftzuAIenPRcWnz1buej2R5uaSAcVuwmZuDs2Vgv+xeTToqcKBczN5aPZ1M4elsJ2UQP+y4ku14HGysKDU9nVLStP+Mnf/iJSpmZ62efsbHwsek4CB43F9nyNvVgf7zSg36ZW8ozdlxhUq62NHvbzagCl7OOodYaOI/nT/tCaXqZVyoeYWMDhIAAFDE+8waA4JZgLzjPxf8UT63EOOBC/xV1z66tkfGJsoEMsb1sh//dYZKuzrQv2NbaAWOSgCaU9mBPn4ZWI/aVPaiwEmb1ds4SL376Al1q+VNrWfvpsi4jIWhdfxcpWtC2IMn6n0/6VhZFro9y+Zz4TTyj5PqUojnBb8AAFAIF4ABgHnhIJVH6j5vH124e8Go1gHSzYAHPgSVKyFdCXIKArld2C9vBMmCMEVQ2RJU1duFVhwNU2d82bQe1WnhgesU9vAJJaemS3/crCOEBz0tP9h2MUIdyLJTtzLbiym+2hJMG8/dpWHNy0vpBI8fzvp9HbuhOSo4hmr6vNjQiJzeBAAAwLMhMwsABcqu4EjpqDCnXx1qVN6d3vjtKO29HCX3zX8jiNpVLUmhUfESBP914jZVLuVCnWuWlvs589r3l0OSZeUglCeh5YTrfJXnfx4Oll9vVFZrW9f/7qdzdzI6O4x7qRKNaBkgx8k1xJYWFnT85gOpPdYnY8vB8MAFR2lMm0Aa3NRfAtt0Femd7eUJcLwvgmEAKCxQZmCgkwMApseLt8YsP0XB4XG0+b3mVMLJ9pn77wqJpCFPp5mxV4N86EpkPF24kzH9T8nobhjdTEb08nPzJLNn4VpaLocYvfwUhdyLoycpaVqBMrcwC/AsJiOEX6lTRvryHr/5kGb2rCEL6c7djqHr9x9LiQNnjrnNmZezPX3XN2N9QP9fD9PBaxnt0K7PeFnqi7/ccJFWDGtEDctnrhfQhY+n69z99GZTfxrfqbJs4+EWz+orDABQ0KHMAAAKDRd7G1o0pIHeH8M3CdAO/rhN17QeNSQA5UTnF+svysKxat4Zb2a5jy6XLjQL9KB7MYl0OPQ+zd9/nYY196dmFTxp/N9nKTwmkWp/uS3ba/FzPElOo9DoxxLIsn9O3VHfv+XCPfJzd5SsK/fK9XCyJV6kcOBqRuA6pWs1Ku5oQ4+TMgPj8hM3yYI49p9Nl2jdu83ol73X6ExYjByrk521PJfF084TXMPMAfJPe65JMPvnsYwWanP71VVnrI3l4eNkqXPW9XMKvhdLD+KTqUlgZkcMAABDQM0sAJgFfT9C54wkZ0A521rLpzh1r11G6meVGlrNrgaM63a5RIBxprZFRU9qX7Uk1S/nJsHiLwOD6K3fj6nrbF+qVpL6Nywrk9E8i9lJpvfztedp5fHb2Y7l7O0YGrHkhASf7M/jYVSldOYnQlci4yionBt/c+ptmktyOVBeevgmTd+U0QWiYklner2RH3Wbe4A8itnS3yObaAXCHNR+uua8PMeoZSepuENDuhwRR280LkvWVpY6yxs4Q8wdI17U2tN36L0Vp+nTzlWk168m7mrR8ft9cn3fx61lrDIAgKGgZhYACh1eZLb9UoQEtZzJfFGPEpJp2dFbEpSNbBWYrZaVs8Y37ydQSEQcfbjyDL3dojx9v/2y1L0yGysLaR/G5Qg8ZY33Y2PaVqBb9x/TmtPPLnNQuNhbU/OKnrTxbEZbw69715S+u9ejH8vtGT1r0AQdgy2Uml8OdgcuOELWVhY0smUgvb7giGSuOSh+ERysB0zcpL59Y2Znrfu5xrnNN3vk+qIh9alVJa8Xej0AKPxi0c0AAIoyzjT2a+BnsOdzdbSld1oFPjNrXM7DSS4dppSU2/9euEcX7sZKve36d5vRiKUn6OStR+pAlnFv29yITUxVB7Js3F9nte7XFciyVSduSzDLHRqUEcFKqcOJmw8pMSVNSik+W3NeksScreYMa1l3J72Oi79XzYA7K2WhHOPX0YXfEHCpBi/oQ5szAMgNk00AAwAozOUQbzXzl4EKv74RJG3Jfh4YJKUJz1Pew0m6I2g6/Xl7GtyknPo2lyrUL5fzyF6ezrZlbHPa9VErsra0oDNhj+j8nRiav++6zv15Utqv+0Jp/9Vo2nclWtqXcXmDgheUPWt62ux/Q7QC7h93XaUWs3bJazKu91Xw4jhdVp+8Q53n7KcZmy7l+FoAALqgZhYAIB/0rOsjF81sMXdQCL4XJwGdMnCBA1zexjWwPKyhQ9VSsijs66cBopuTrWSGW1bylC4HrE89HxrStJxkRCevu0ARsZl9c1l17+KS4WQv1ygt9cM95x1Ud3LgAJelPq2DGLr4eLbjD4mIlwVdw5eeoODwWOmjW6q4PQV6FZNyiSFN/WU/DpB5ARwfZ1JKGj1OTlMfOy9K43rk3w5kBtHcC1iXb7ddzni+/delY8XAxmVl8R8AwPMgmAUAMBJehMWLzKqWdqGBjcpSWe500Lgs/bInlFpX9pL7NINfrv3tUbuM3G5c3l1de8tDJzgD3LF6aXqpWimKik+iCX+fk04BR6/fp/fbV1Q/D3c44Ewrd3NgPeuUoak9qktXBc6oKgGygnv5cnC770oU9fn5EF2NjJftnLXVxIEmZ4i/3ZYRuI7vWFmeS+nqwDacDact5zNLEBjXCCudKfgr9wquW7YE2dlkflDIwfCt+wnSU7htZS9ZiPeiFh+6QTeiE6R1Go9h5qBcX1cj4+heTBI1q4BODAAFERaAAQAUQNei4mnrhQgpV1A6MXBnAl7EpRn06mPNqTs09s/Tcv3PtzN71y7Yf52mbriote+ZyR2o1hdb1bdLudjLgrHbWTKq3N6M64FvcNBZpST9+kY9mvjPeVp+9Jbc7+5kS/cfJ8t1LonoW9+PPlp1Rm6XLm5PAxr6kYOttbw+Z4qVLHFW77YOpI9eqkQ3oh9Lb98aPhnfO3dx4PNio9GlgcspuJ0Z1/tqTmTjfatN/lerrvfIxHYS1D4P1xNX/myLXN/6fgvpKJEVL8D75K+zsvCPA3AWEZtIp249lDcbGGYBkHtYAAYAYOZ4CMPIVtrZQ12BlD561Ckji7o4g8sTyhTda3vTymNhVKFkMQnIOBgr7qD90f6yYQ2l5peDs1F/nJSAMuZJCsUnpUogy0HrzF41JGDjccUcW/IABx4aMX9fKDUL9KQPO1TUClZ5EdjsrRllBUy5z97Gkj7qUEnGESvm7roqi+Y4u8yZ6T3jWsuEtU4/7JWgfuHg+uq2Y91/PCBfJ/5zjjaMbq5+jtNh2iOLua73UGg0NfR3pynrLlDbKl6S5dZl07nMBXeHrt3X+hlwkLzk8E0pp3iUkEJHbzyg0OkvSyb57cXH6cztGPqiWzUapFHvnFec4X6YkCxvBPhnydPrACADygwAAIoALk3IyqOYHf37fots24vZWUuwWsLRhsp7ZgTUxTyL0cYxzaU0YNa/IfTL3lDZPrpNoDwP8ynhKAMqGD/u1SBfredVpqNxScV32y5ny8ZycMk9alceD6PLERnlDYwDWcYlFjwp7Xp0PD1MSJHFaoGTNlPzCh5aC+TO34mVsoJedX2kNduxGxkdHDTtCo6i0KjH0umBL7s/aiU1wRy88nNzy7L9V6JoR3CkVt9gzUVx3OaMO1Ro2nMlilpX8pJAlnG9MGeTuWyC3zwkJPPwDgu9ssKa/Ya5G4amy9M6qTP2AEUdglkAANDyw2u16b87r8rEMU0ZLbMsJEjkrCsHsTyuV1/K+F5Wy8eV3vjtCNXwcZX64SOh96V+mM3tX1dqad9pFSBtvYYtPk6JKRmL13i/S/cy63IZB7V80fT52gu041IkzR8UpB4VzP12uWyCa4I5o6qp7bd7yN/DSV0jrMvfJ29TaHQ8rXi7kWRKswaybNPZcGrAgzCe4v7Dg347KhltroHm0g7OQG8Z20KGW0zdeJFaVPDIMTPMuFwhq9sPE+QNA7+5WHX8Nv11MuN86erhyyUr9+OTtbLyAIUJamYBACDX+KN7LjF4kWlevLCKA2Lu1vAsPKls2ZFb9MeRjHrcrLh+lxe4lXF1oDuPdHdL4I5pOz5oSd6uDtTnp0NavW9z69tXa0mgvPFcOPWsW0bqmPmjfw7Aa/u60pfdq8mUtmfh8gPuCsGjh9k3fWpRr3o+tPdyFH21JVhqbTk45RIKHsDx/XbtnsTlPZ2kU4SznbW6M0bLip70+5sN5DoHuXxO+NzUnLKV4pJSae2oplTLN7OW2JD49bhUhTPOpYs7SMu22Ccp5P40aw+QnzWzCGYBAKDAi0tMoXpTt0t7MQ5MuZsC1+6ybe+3oMOh96lNlZL0x+GbMhWNSw+4A4MyHpgDzDcaZ5QipKal0897Q6VrApcTfPdqbWrx9a4cX5sDVD83R2lxltW6d5vKYjNenNfhu72yrUmAuzobnBMOvGv7uaqHYLg62sio37cXn6BDoRmPrVGmuNQwZ+0koamat4sM52Dexe3p4IS2cp0X1nH2mafE8ZhhxosJP+tSNdtz8GQ7Lo/g5+Lxxjnh/dJUKq1Fd4pVx8PUQzy+71tbSjtWHAuj1SOb5FsAXRBxeQ5n0rn7iK4R0lxnzb+/jrb4YPx5EMwa6OQAAEDBsTskki6Fx0kG0tbagjp+v08CJc1xvNx9gIc1cJDKwSd3hOAFWLo+Yr8SESfBm4uDNflPyBzH275qSXWdrtLO7Nu+tenB42Rq9fUuWUDGyrk7ymAKXvzG2dmqn2/RqgPmGmHfEg6y8G5U60ApN+CaWS430NW9gTOx3LdX6QecF+e/eImcbK0oaNp26SahZK2VgRw7Pmwpx7v1wj3JqnN7NS4Z4UV3XP7AGWHuCsHt0vgNAbckU5GKhjYrT93m7pfj3jimGdlZW6kD3A3nwmnJoRt07EZGOUSP2t7qEc18LDy2eVjz8gZpsaa0dePFePyGwM89758MaOJM8u6QKGoa6K53oMnHsv5sONX1c5V6ccYlMfy7wx01uP47a+1zq9m7yM3JjjaNaYYuF8+BYNZAJwcAAAou/ljbzdFWhky8qBqT/5WP4pX2ZMeuP1APk+CaXu7Ny7irQ2RsEt1/nEQVSjpLQKVoPGOH1rje6a/UoP4N/bQCJl78xS3K/jl1R7ZxsMm1vB8+bVvGvJzt5OP5S+GxElC+Vt+PUtPT1RnWrDhGLOGY0QqNM9B7QrQXrmUNmJsGetCA+UckQ3hkYlt65ceDOZZnKMa0CaQ5O6+qb7/eyI8+7liZ/jp+m77M0t4tpzrs7rUzSjJmbw2hAE8nademLw6a+/16WN6s8OO4YwUP6uDAmssacnL8xgM5jzzeWlemlOuJ+WfIz8dT6Lgjx7iXKmfbjztW8JuQCZ0qq5/njyM3adI/56lSSWf1Qspy4zdmy5IzztZyac4X6zPO1fYPWlCgV966kxQVsbmI15DnBgAAs8QLtgzl54H16M3fj9F/etSQj/a5XywHfiuP36bhLTMzbCVd7OWii1L2wDj45XpaTcrH851rlFYHs33r+0qXA56ApgSUzQI96MOXKslit661vOVxHMz9tCdUAjMFlxBwuQRPhNtzOUpKG3jhmy5KPTGXYPCFcQlGg//sUHewUD4m5/ZfnKHlWmUl26oZyLKlh2/JRV8LD9yQdmIzNwerv3deDMdt4/gjeS754NZwygI2XojH31On6qWkzpmHcRy5ntGV4sztjDpjzpRzgN880IPq+7tJUMmL9PicOtvbyPH3n39EAuh7sYmShU5LV0lQzTi7ywEy1y4npWZkw3/ddz1bMMvnnI+bpatUNLlrNbn++9OBI9w6jrPZPMZZoXSr4Ozt3J1X6ZunE+4UR68/fKFgNiE5lT7+66z8nLrW8n7u/vx985spru0ujFAzCwAAYAA8MGLC6nM0tXs1Gvi0PjenwGLWlmDycXOk1xv6ycfNSpaPg87f36yvM9Dhx/Giqnbf7qEyJRxkQZcykGHy2vP0+yHtDg3DW5Sn1afuSEaR9z156yHN2BwswV1Wk7tWpfrl3OiHHVfog/YVJfBTMpdtv9mj9zngw1HqlPPy5mTnhy2lf3Hv/x2UTLOjrRX9805TmUin2X+Yu1JwgKpL55ql6cf+deU5uN9xVsuGNqTIuCRpm6bZbk05/rn96lKbyl4SkHJpwNtLeCJeZt3ylrHNyd3JjprM3CHt4nThMdWb32suC/f4nGallK6wRwnJ9Ou+UOkSorTCywm3hBu36qyU3CjlLjdmdn7mY/iNUOf/7pfx1L8Nrk8VSxbTmaUuaFBmYKCTAwAAoC/OwkXEJkm/2ry4ef+xZCF1LbDSFJuYIlPTNGs7+WNsDqT7BPlSLZ/i0lmBRxlz9pL76SqTyXjB2ahlJ2UAxfJhjaStV/OKHtSlZs7ZvQHzD9OBqxmL0vhxxR1sKTo+Kdt+nMH+eU9G/2HGo4g5W8wT5Lg1mT44a70jOELdik3Xa/Rv4Ee+JRyp5/8OZhuIofAp4SBT6zg4LevmKAFyTlpV8pSa6llbMkYzMz6Hi99sKP19lQV5nLHmMhLO3HN9Mf+s84qPb/8nbeQ6lzhwtw7u67z7o9Yy2pnP4y97r0lPZm6Rp/ysVxy9pe6AofjnnSZSvsKfGPDvHpdWrD19V8ZacymG5uJEZWHh+tHNsh3TiZsP6KstIVL2ktcBLYaEYNZAJwcAAKCwBdxrTt+RwKdJgIdejwm+FyuL7djeca0lEPtp7zWq6OVMVlYWVLW0i0xAK+fhSJU+zRj9y65Nf1lqXDk454Cq908HyYIsaMHgIBq88Jg6Q8xBHGeCNTtAcK/eYS3Ky4IqTevfbaYeabzlfDiNWHpSAk9lSEVWdfxcJajWnDiXlTKAovyEjaS5Lo+PSSnr4HZsgV7FtFqu8QLAka0C6JO/tYNLXVnaOn4l1KOeFVw3ywH0Vo3Fhrqy2xNfrkwtK3pJnfbSwzdp07l7Ob7WB+0r0s97rtHj5DRZzLZgUH15Y/PpmvNa+538rL0EuoqYhBSqPXWrvC5nbre+35JMDcGsgU4OAAAA8MQ0/lg7RV1vmhNu0cUjgn8dFJQtWA6NipdgkYNCHvrgbGcjvXoblncjWytLGrPilHzsz1Pl3mtbQUoo3l12Unr68shlrq3lbLNSWsEOXI2WoDM4PJY+WX2W3mrqL4GtUpf7UYeK1CTQg3rOOyi3uQQk+F6clC/wa3HnCqXrAGe2swacbEBDP/rPKzXUmW2eLsclEZ92qSq1xhvO3qVpGy7lWPbAx8yL9KZvyqi71dVW7Vm4IwS3RMspW834++FOGbq280JFrukd1txfMsAc6P41ojEFPR3uwbW+XLOsKXhqR7px/zFVLpUZJ3GrNxd76+f+DhgKglkDnRwAAAAwHq4Lzpg0l1nvyXLb1osntHGQyQv6nOysqOXXu6VX8Z5xramERkZS0/34JPl4vk+QD41efkpadXFpwcrhjfUaDsLBPj/+M40sKC/u40CYg+7hS07oDGI50/3H0Ib04cozWjW+XNLBwTKPV36eZUMb0qCFR9U1vFw3rPQw1tznf3uuqet/uW6XS0ze/zOzk4airLujTK/jzPP77SpKNp8XnPH7CK5h5t7L+Q3BrIFODgAAAJi/yLhECfQ0W6k9Cy/84tHFnEVWeurqq+t/98uEOe6Gwe3LspZqcHsvXoinBKzHJrWTKXicreZAeFdIlNxX06c4DWpcTt22jVu9ta7sKaUGP+7K6EihuPKfTlL/PHndeem28HqjsrT+zF0JypV2b/s+aU0zNgXToqddGPJicJNyslhQMzueX9CaCwAAAOCpZ00204U7GVTzzqjNzUubt/1XomU8saKsW2YbOW4dxm3Hjl5/ID2ElXHOPHjhlzeCqMKkzXKbW4bxc4THPKG7MYmSMeb6Yx4IouAs9oCGfrKdexrzcyuZ7XZVMhb9sXfbBEpQHuCVvVsC1zy/XKNUtrpibvm2+3KUdMPgBYfcOu2jDpUK5LAH9JkFAAAAMBDuSPFqfd9swTG3HWMcvC4b1khKKLKWT2h2suCuCezdNhW09mldyUs6DlT1dqEAz2JSU6vQLNHg11w4uD5duhcrHSAYj2VWcOuwu4+eyOI0XoTHnRN4ARjXPQ9p6k/lPJzkGLk/sZOdtdaCsYLG5H1mHz58SOvWrSNnZ2fq2bPnM/dNTU2l+fPnZ9veunVrqlSpkl6vhzIDAAAAKKi4/RYPw+BAlANKQ0pMSaM3FhylSqWc1VPtCiqzqJlNS0ujoUOH0r///kuOjo7k6upKx49rt+DIKj4+XoLeV155hby8MqaEsEGDBlHjxo31el0EswAAAAAFm1nUzHIM3aJFC5o3bx5NmDCB9u/fr/djP/74Y2rUqFG+Hh8AAAAAFHwmC2atra1pyJAheXrs3r17KSQkhAICAiQja2VVOGcNAwAAAEAhWwDGgeumTZuodOnStG/fPvLw8KB//vmH/P39de6flJQkF820NQAAAAAUDs8eAF3A2Nra0uHDh2n37t20fPlyyc7yNq69zcmMGTOk5kK5+PpqrzAEAAAAAPNldsFsUFCQ+raTkxONHj2a9uzZQ4mJusfIcT0uFw8rl7CwMCMeMQAAAADkJ7MrM8jK3t5eOiNw+QBfz8rOzk4uAAAAAFD4FOjMbHJyMv30009STsBu3LhB6enpWvssW7ZMFoJptuoCAAAAgKLBpJnZlStX0oMHD+jcuXMUFRUlgSvjGljudpCQkEAjR46khQsXylCEgwcPUq9eveill16SvrS8EOzs2bO0atUqU34bAAAAAFAUg1nOuN65c4cqVKggl9OnT8t2LhvgYJbLA4YPH66e7tW/f3+qX7++dC/g4Ldv3760evVqcnNzM+W3AQAAAABFdZytsWECGAAAAEDhidcKdM0sAAAAAECh7maQW0oiGsMTAAAAAAomJU7Tp4CgyAWzcXFx8hXDEwAAAAAKftzG5QbPUuRqZrm11927d8nZ2ZksLCyM8s6CA2ce1vC8mg/AeS8M8DuP816U4Pcd570oiTViTMPhKQey3t7eZGn57KrYIpeZ5RPi4+Nj9NflHzqCWePDeTcdnHuc96IEv+8470WJi5FimudlZBVYAAYAAAAAZgvBLAAAAACYLQSz+YwHP0yePFm+gvHgvJsOzj3Oe1GC33ec96LEroDGNEVuARgAAAAAFB7IzAIAAACA2UIwCwAAAABmC8EsAAAAAJgtBLP5aPv27dSnTx9q1aoVvf/++xQVFZWfL1ckHD9+nIYOHUqNGjWiAwcO6Nzn8OHD1L9/fznvI0eOpFu3buVpH8iQkJBAc+fOpZ49e1KnTp3o008/pejo6Gyn5+LFizRkyBBq2bKlfOXbedkHMm3ZsoUGDRpEbdq0kfO1f//+bKfn5s2bNGLECDmnAwYMoKNHj+ZpH8juzp071KJFC+rcuXO2+yIjI2ns2LHyN+TVV1+lnTt35mkfyPwd5b/rWS/Hjh3TOkUxMTE0YcIEat26NfXo0YPWrl2b7RTqsw9o478Jb775pvyt+eijj+jBgwda9z958oSmTZtGbdu2pS5dutDSpUuzPIN+++QXBLP5ZOPGjdSxY0eqWbOm/GKcPn1a/igmJibm10sWejNnzqThw4dT1apV6ciRI/Tw4cNs+/B2/ge7TJky9PHHH8u0tyZNmtD9+/dztQ9k6tChA12+fJlef/11Gj16tLyJaNCggdYfu6tXr8o55Kl648ePl218+9q1a7naBzLNmTOHFixYIOef30DwFBwOijZs2KAVLDVu3FjeKPM59fLykr8z/KYvN/tAdmlpafKGl//OnDhxItsbvGbNmtGFCxdo3Lhx8jeJf07//vtvrvYB7UCI/zZPmjSJvv/+e/WlUqVKWj8TPof8poATRM2bN6fevXtrBU367APali9fLn8T+N/Ezz//nAICAuTfWk2cmPvjjz9o1KhR1LVrV7l/9uzZud4n33A3AzC8mjVrqt566y317YcPH6rs7e1VP/30E053Hj169Eh9LvlXd/369dn26dChg6pz587q20lJSSovLy/VlClTcrUPZIqLi9M6HbGxsSpbW1vVggUL1NvefPNNVe3atVXp6elym7/WqFFDNXTo0FztA5keP36c7XS0aNFCNWjQIPXtiRMnqnx8fFQpKSnqbW3btlV16dIlV/tAdp9//rmqZ8+eqhkzZqhKliypdd+cOXNUTk5OWv9vDBgwQBUUFJSrfSDTpUuX5O96WFhYjqdl5cqVKktLS9Xt27fV2z788EOVr6+v+u+KPvtApgcPHqicnZ1V06ZN09iqUj158kR9fe/evfKzOXXqlHrb119/LY9LSEjQe5/8hMxsPuCPYM+ePSvvTBSurq7yzodLDyB/xtqlpqbSnj17tM67ra2tZMiV867PPqCtWLFiWre5v6CNjQ0lJyert+3YsUM+VuKsK+OvfI41z6k++0AmR0dHrdNx7949yZDXqlVL65zy7661deZk8u7du8v29PR0vfcBbXv37qWFCxfSL7/8ovPU8Lnjj7A1/9/gc8rZ7kePHum9D2THH3Xzx9TvvPMOXbp0Kdt5r1u3rmQQNc9pWFiY/L+h7z6Qac2aNRQfHy9lSJrs7e3V1/mc+vj4UO3atbXOaVxcnLpkSZ998hOC2Xyq/WH8saAmvq3cB4YXERFBSUlJzzzv+uwDz/bDDz/ImwIOkBi3quZ/KHSdU6UWWZ99IDt+w8B1g3Xq1KHAwED5B4drMBX8O6vrnPJHtkpdsz77QCYuN+KSGi7xcHd313lqcjqnTPl91mcf0KbUhnNpHv9+8hs3fmOhz3lX/n7rsw9k4nUL/v7+dO7cOakvfumll2jixIlaZXfmcN4z36qDwaSkpMjXrBMyHBwc1PeBac47fjYvZtOmTfKHbt68eVSuXDl1jRpn+HSdd97O93Mw+7x9rKysXvDoCh/OgHPd4OPHj2nbtm00a9YsatiwofqNBP8+6zqnyn367gOZOJji2r/27dvneFpw3g2vfPny8imN8skNLzblemVe18ALdpXz7uzs/Nzf9+ftA5l4HQ/X1X/44YdSm8+fVM6YMYP+/vtvOnXqlHxCpOv3XcncPuvvTNZ98hOC2Xzg5uYmX7OuBuR3Ojm90wfjnHf8bPKOg6levXrJQrxhw4apt/PH1/yPh67zzqUhSpCqzz6gjf9h58ws449e+ZMFXiCjBLP8+6zrnPLjSpQoofc+kHle1q9fL5lw5bzzAlE+f3z7iy++kMxVTueUaf6ted4+kImDqKzatWsni+cUOO+G5+bmJmUGv/32m7qEKSgoiEqVKiXdVLiLDe/Di9g1Kb/bmr/vz9snP6HMIB/wSkD+BzprSxGuG+E/kpA/XFxc5NxnPe+8QlY57/rsA9lxxoTrn7788kt5B58V16g975zqsw88W+nSpbVKA3I6p7wCXKm51WcfyMB/tw8dOiSfPCir6bmum9+I8fV69eqpz2nWbhB8TvkfdD8/P733gWfjOnEnJyet33cOmLjMSfOc8icY1apV03sfyMSBK9MsEfDw8JDzpXQM4nN65coVaXmmeU6ZUiOrzz75Kt+XmBVRY8aMUfn7+6siIiLk9qJFi1RWVlaq8+fPm/rQzN6zuhlMnz5d5enpqQoNDZXbmzdvVllYWKh27dqVq30g086dO1UODg6qWbNm5XhaFi9eLPucOHFCbh8/fly6dyxZsiRX+0CmH374QRUdHa2+HRISoipTpoxq+PDh6m1bt26Vldvbt2+X21euXFGVKFFCVhHnZh/Ima5uBqdPn5Zzunz5crkdHh4uq+V51Xxu9oFMS5cuVQUHB6tv898JV1dX1ahRo9Tbbt26JX8zvvvuO7kdExOjqlatmnSJyM0+kCkxMVFVrlw51eTJk9Xb5s2bJx1r+G+O0knIzc1N9cknn6gf06xZM+mKotBnn/yEYDYf2+p07dpV/vEODAxUFStWTLVw4cL8erkiE1Q1bNhQWttwMFupUiW5zS1wFNx+6I033lDZ2dmpKlasKH/Usv6jrc8+kKl06dJyjvhca140zzvjf6T5DyCfU/6q6x9tffaBDBwE8T8yfK74wn9L3n77bVV8fLzWKZo5c6b8fHgf/p0ePHiwKjU1Ndf7gP7BLPv111+l9Rb/feefTffu3bO1INJnH8hw8OBBVd26dVV+fn7q31NOCmm2iGJ///23BLmcLOJ/V1u1aiXtpXK7D2TidloBAQGqsmXLyu+qh4eH+k2Y5r+//P8BvyHjc8s/q6xt1PTZJ79Y8H/yP/9bdPEKbq6TqlChgtbHJZB7XH+jq7UKfzyS9WM7/ngqPDxcFhXk1NJLn32ApGG8rgJ+Xeedf9d55WrZsmVzrJPSZx/IwH+eQ0ND5SNTPtfKQpas+KM93o9/JiVLlszzPpAd18zy3wr+GDUrXpjHf5M8PT2lLZEu+uwD2lPXYmNjZYW9ZnuorIuWgoOD5e8275fXfUD7bw2fL0tLS/k3kcsMsuJ/B7hdGv8d4phGF332yQ8IZgEAAADAbGEBGAAAAACYLQSzAAAAAGC2EMwCAAAAgNlCMAsAAAAAZgvBLAAAAACYLQSzAAAAAGC2EMwCAAAAgNlCMAsAYCbOnj1Lu3fvNvVhAAAUKNamPgAAANDPsmXL6Pjx49SqVSucMgCApxDMAgAUwJGeZ86ckVGc9erVk7GeISEhMiYyIiKCVqxYIfs1a9ZMPSKV7+N9eFxtnTp1tMZRnjp1iuLj4+W5+PrDhw+pefPmGOMMAIUCglkAgAJkzpw5NGnSJGratCklJSVRZGQk/fnnn3Tjxg26cuUKPXjwgNasWSP78sx5Dw8Pev311+nAgQMUFBRE169fJwsLC1q/fj2VK1dO9vv9999p69atlJqaSr6+vhQVFUV3796lLVu2yGMAAMwZglkAgAJkypQp9Ntvv1GfPn3k9s2bNyk6Opq6dOlC+/fvlzIDJTPLPvnkE8nWhoaGkoODA6lUKnrrrbfo3XffpQ0bNqj348ztP//8Qz169JB93njjDRoxYoQ8HwCAOUMwCwBQgHBJwYULF6hnz55kZWVFZcuWlUtOFi5cSL1796aNGzdKkMqXUqVK0cqVK+U6Z2lZYGCgBLKMt3388cdUs2ZNunbtGgUEBBjt+wMAMDQEswAABcjixYslq8rlBi1atJAMbf/+/dVBqaaEhAQpGbh48aKUH2jiTG5ycjLZ2dnJbaXkQMElCkrmF8EsAJgzBLMAAAVIu3btKDg4WGpfuab1/fffl4VdX375pc4srq2trQS7b7/99jOflxd96brNNbcAAOYMfWYBAAqIlJQUWfClZE5HjhxJ/fr1o8OHD8u2YsWKUWJionp/S0tLat++Pf3666+UlpaWrSOCptOnT8siMsXq1avJ09OTKlasmM/fFQBA/kJmFgCgAAWzTZo0oU6dOlHt2rWlhIA7EXz99ddyP3cemD59upQgeHl5SWuu77//nlq2bCnXOUObnp5Oe/bskcCXSxYULi4u1LFjRylh4AVj/Jz8WM7uAgCYMwsVrxAAAIACgfvBLlq0SDKpHIB27dqVWrdurTU4gYPVmJgYKUFo2LChlAzwY3hCmKurq+zfrVs39WPGjh1LV69elc4Ha9eulf15MRg/NwCAuUMwCwBQyCnBrGarLgCAwgI1swAAAABgtlAzCwBQyNWtW1fG3AIAFEYoMwAAAAAAs4UyAwAAAAAwWwhmAQAAAMBsIZgFAAAAALOFYBYAAAAAzBaCWQAAAAAwWwhmAQAAAMBsIZgFAAAAALOFYBYAAAAAzBaCWQAAAAAgc/V/a92RMK+AzAoAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 800x400 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(8, 4))\n",
    "plt.plot(train_losses)\n",
    "plt.title(\"Char-RNN training loss\")\n",
    "plt.xlabel(\"step\")\n",
    "plt.ylabel(\"cross entropy\")\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "cell-006",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-07-31T21:59:38.750033Z",
     "iopub.status.busy": "2026-07-31T21:59:38.749926Z",
     "iopub.status.idle": "2026-07-31T21:59:38.786511Z",
     "shell.execute_reply": "2026-07-31T21:59:38.785968Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The mace often albwoun building texts . It . The Conduabor , and [ @,@ 000 god , 1s ) and won sertic gold dosent exteck operationsious lack tradicated by they was nubind muling continaliginal . The asiggued that worford opecting creatities , the muning its rifle thar , and withlin bypart an troingne of Aprubior , ifter activiled the shiption d. spection . He instinus mopernor plales , and Wather , ḏnyRual in an acterornarder by fuggulatur is reill . In and terall anaway . \" A Griate recially sujical towner owerificating buildings tradition in the samited in the 72 @,@ 0000 been larnunes with su\n"
     ]
    }
   ],
   "source": [
    "start = torch.tensor([[stoi[\"T\"]]], dtype=torch.long, device=device)\n",
    "sample = model.generate(start, max_new_tokens=600)[0].tolist()\n",
    "print(\"\".join(itos[i] for i in sample))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cell-007",
   "metadata": {},
   "source": [
    "## What To Look For\n",
    "\n",
    "The model usually learns punctuation, line breaks, and some short-range spelling statistics before it learns anything like semantics. That is exactly the point. The recurrent state is carrying local context forward step by step, and you can see how far that alone gets you.\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}