![](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAxoAAALnCAIAAACBWaZ2AAAACXBIWXMAABYlAAAWJQFJUiTwAAAgAElEQVR42u3de5BedX348fPc9xYxlyXb1pBQrXE0yTjCmJJwGWFD2wQDqENM4kCnUistWS521HaqFKrTaZXKvZXLWCOb0Hgb0CWIIZUxMONQrZNQxo4wGyZpZ3NrFLLsPpdzzu+Pb91hbH9T2M34fPfJ6/UHE/YPPX4/z/me9znn2ZgkAADMQCHPc6sAADBtRUsAACCnAADkFACAnAIAkFMAAMgpAAA5BQAgpwAA5BQAAHIKAEBOAQDIKQAAOQUAgJwCAJBTAAByCgBATgEAIKcAAOQUAICcAgCQUwAAcgoAADkFACCnAADkFACAnAIAQE4BAMgpAAA5BQAgpwAAkFMAAHIKAEBOAQDIKQAA5BQAgJwCAJBTAAByCgBATgEAIKcAAOQUAICcAgCQUwAAyCkAADkFACCnAADkFAAAcgoAQE4BAMgpAAA5BQCAnAIAkFMAAHIKAEBOAQAgpwAA5BQAgJwCAJBTAAByCgAAOQUAIKcAAOQUAICcAgBATgEAyCkAADkFACCnAACQUwAAcgoAQE4BAMgpAADkFACAnAIAkFMAAHIKAEBOAQAgpwAA5BQAgJwCAJBTAADIKQAAOQUAIKcAAOQUAAByCgBATgEAyCkAADkFAICcAgCQUwAAcgoAQE4BAMgpAADkFACAnAIAkFMAAHIKAAA5BQAgpwAA5BQAgJwCAEBOAQDIKQAAOQUAIKcAAJBTAAByCgBATgEAyCkAAOQUAICcAgCQUwAAcgoAQE4BACCnAADkFACAnAIAkFMAAMgpAAA5BQAgpwAA5BQAAHIKAEBOAQDIKQAAOQUAgJwCAJBTAAByCgBATgEAyCkAAOQUAICcAgCQUwAAcgoAADkFACCnAADkFACAnAIAQE4BAMgpAAA5BQAgpwAAkFMAAHIKAEBOAQDIKQAAOQUAgJwCAJBTAAByCgBATgEAIKcAAOQUAICcAgCQUwAAyCkAADkFACCnAADkFAAAcgoAQE4BAMgpAAA5BQCAnAIAkFMAAHIKAEBOAQDIKQAA5BQAgJwCAJBTAAByCgAAOQUAIKcAAOQUAICcAgBATgEAyCkAADkFACCnAACQUwAAcgoAQE4BAMgpAAA5BQCAnAIAkFMAAHIKAEBOAQAgpwAA5BQAgJwCAJBTAADIKQAAOQUAIKcAAOQUAAByCgBATgEAyCkAADkFACCnAACQUwAAcgoAQE4BAMgpAADkFACAnAIAkFMAAHIKAAA5BQAgpwAA5BQAgJwCAEBOAQDIKQAAOQUAIKcAAJBTAAByCgBATgEAyCkAADkFAICcAgCQUwAAcgoAQE4BACCnAADkFACAnAIAkFMAAMgpAAA5BQAgpwAA5BQAAHIKAEBOAQDIKQAAOQUAIKcAAJBTAAByCgBATgEAyCkAAOQUAICcAgCQUwAAcgoAADkFACCnAADkFACAnAIAQE4BAMgpAAA5BQAgpwAA5JQlAACQUwAAcgoAQE4BAMgpAADkFACAnAIAkFMAAHIKAAA5BQAgpwAA5BQAgJwCAEBOAQDIKQAAOQUAIKcAAJBTAAByCgBATgEAyCkAADkFAICcAgCQUwAAcgoAQE4BACCnAADkFACAnAIAkFMAAMgpAAA5BQAgpwAA5BQAAHIKAEBOAQDIKQAAOQUAIKcAAJBTAAByCgBATgEAyCkAAOQUAICcAgCQUwAAcgoAADkFACCnAADkFACAnAIAQE4BAMgpAAA5BQAgpwAAkFMAAHIKAEBOAQDIKQAAOQUAgJwCAJBTAAByCgBATgEAIKcAAOQUAICcAgCQUwAAyCkAADkFACCnAADkFAAAcgoAQE4BAMgpAAA5BQAgpwAAkFMAAHIKAEBOAQDIKQAA5BQAgJwCAJBTAAByCgAAOQUAIKcAAOQUAICcAgBATgEAyCkAADkFACCnAADkFAAAcgoAQE4BAMgpAAA5BQCAnAIAkFMAAHIKAEBOAQAgpwAA5BQAgJwCAJBTAADIKQAAOQUAIKcAAOQUAAByCgBATgEAyCkAADkFACCnAACQUwAAcgoAQE4BAMgpAADkFACAnAIAkFMAAHIKAAA5BQAgpwAA5BQAgJwCAEBOAQDIKQAAOQUAIKcAAOQUAAByCgBATgEAyCkAADkFAICcAgCQUwAAcgoAQE4BACCnAADkFACAnAIAkFMAAMgpAAA5BQAgpwAA5BQAgJwCAEBOAQDIKQAAOQUAIKcAAJBTAAByCgBATgEAyCkAAOQUAICcAgCQUwAAcgoAADkFACCnAADkFACAnAIAQE4BAMgpAAA5BQAgpwAA5BQAAHIKAEBOAQDIKQAAOQUAgJwCAJBTAAByCgBATgEAIKcAAOQUAICcAgCQUwAAyCkAADkFACCnAADkFACAnAIAQE4BAMgpAAA5BQAgpwAAkFMAAHIKAEBOAQDIKQAA5BQAgJwCAJBTAAByCgAAOQUAIKcAAOQUAICcAgCQUwAAyCkAADkFACCnAADkFAAAcgoAQE4BAMgpAAA5BQCAnAIAkFMAAHIKAEBOAQAgpwAA5BQAgJwCAJBTAADIKQAAOQUAIKcAAOQUAICcAgBATgEAyCkAADkFACCnAACQUwAAcgoAQE4BAMgpAADkFACAnAIAkFMAAHIKAAA5BQAgpwAA5BQAgJwCAJBTAADIKQAAOQUAIKcAAOQUAAByCgBATgEAyCkAADkFAICcAgCQUwAAcgoAQE4BACCnAADkFACAnAIAkFMAAHLKEgAAyCkAADkFACCnAADkFAAAcgoAQE4BAMgpAAA5BQCAnAIAkFMAAHIKAEBOAQAgpwAA5BQAgJwCAJBTAADIKQAAOQUAIKcAAOQUAICcAgBATgEAyCkAADkFACCnAACQUwAAcgoAQE4BAMgpAADkFACAnAIAkFMAAHIKAAA5BQAgpwAA5BQAgJwCAJBTAADIKQAAOQUAIKcAAOQUAAByCgBATgEAyCkAADnF7HbkyBGL0GHGx8ctQidptVqtVss6OEmRU0Rq27Ztp59++i233GIpOsamTZv6+vqeeuopS9ExLbVo0aJFixa5AHeM/fv39/X1rVmzxlKcUgp5nluFTj2lzzzzzPDnPXv2rF692pp0QB9v3rw5SZL+/v5/+7d/6+/vtyYd0Mfbt29PkmRwcPC73/2uBemAPv71X//18FrgzjvvvPbaa63JKcLTqc706KOPnnPOOeFye8YZZ5x77rnbtm2zLLN6j77rrrs2b968bNmyJEmOHDmyYsWKffv2WZnZa3x8fKqlkiTZtWvXmjVrPKOa7TexZ511Vmipt771rVu2bLnlllu8yZVTzFbXXXfdunXrTj/99MceeyxJkr/8y78cHBzcvHnzmjVrnNiz9Lp71llnbdmyZePGjR//+MeTJPn+97+fJMmKFSvuuusu6zMbPfXUU295y1u2b98+PDwcfjIyMrJr1663vOUt3uTOUnfdddeZZ555+PDhsPGeddZZQ0NDN91001RgIaeYTY4cOXLHHXfceeedP/zhD+fNm5ckSa1W++53vzs8PLxr167jx49bolnnySefPHz48MjIyLZt20qlUpIkb3rTmw4cODA0NPTZz37W+sxGO3bsOP3000dHRzdt2hR+snbt2sOHDy9btmzHjh3WZzb67Gc/u3HjxgMHDixdujT85Pbbb9+7d+/hw4efeeYZ69PxfHeqk4WvTw0PD09t2cx24etTo6OjS5YssRodsgsXCkmS2Io7bOPduHGjr1icUjydAgCQUwAAcgoAQE4BAMgpAADkFACAnAIAkFMAAHIKAAA5BQAgpwAA5BQAgJwCAEBOAQCcbGVLcFLkeZ5lWZ7nUR3V3Llzt27dunLlylarFe3SFQqFYrFYKBQinGmaprEd1cqVK7du3Tp37lwznYYsy7Isi+2otm7dmiRJzAMNMy2VShEeWJqmcW68ixcvjnympVIpwpN09irE9kGcdbIsC+ezlZzJTh0262IxiselaZqGODbTaQ80SZJisRjPfp2maYhjM53JTEulUiRRFe52Qhyb6Uw23nCeWo2Z83RqplqtVpZl9Xo98huRyG+SarVanueVSqXtV9/Qx61Wq9FoRPh0araoVCrVajXLsmq1GklLNRqNer1uNNNWq9UqlUqe5+Vy+y8cUxtvs9k0mukpFovVajXsupHcysqpU1e43I6Pjy9YsMBqTMPo6Gi5XK5Wq2ma9vX1ZVnW9vukcOnt7e01nWnPtFqthocHvb29WZa1facOLTVnzhzTmd5Ai8VipVIJNxtdXV1tP6Tw5HhiYmLu3LkGNL2ZlkqlcMMT3svLKTnVZlmWTd0bPfKIB86vz/r1hfAEqNVq9fX1NZvNtn8/I2zTjUbDQGcy0/DwoLe3N03Ttu/U4X1QeC5lptMbaKFQmPrOWVdXV5qm7T1P018w02nPtFQqpWna09NTr9dLpVKe575HNUOC9CRcer0SmraXX375Zz/7WdgfwwWvvd8UDjP13naGM33llVfCl8+azWbbv4IW3t6ay0wG+tJLLzWbzXCGhrdsbb+PbbVavjI1w403nKHhu78R/oqGnDrlRPgLfbOI7z106kkR252Poczk0vtLa9je9Qz/7WY6ExMTE1HNVE6B6xx0OM/2zBQ5BQAgpwAA5BQAgJwCAEBOAQDIKQAAOQUAIKcAAJBTAAByCgBATgEAyCkAADkFAICcAgCQUwAAcgoAQE4BACCnAAB+NcqW4GRZv75gEQwUM8VMkVNM0+jo6LFjx9I0nZiYsBqvXalUivPADh48WK/Xjx8/3mg0Go2GSXXGSdpoNF5++eWJiYk0TS3IbD9Jw0xPnDhRr9dPnDhhUq9dtVq1CHIqxr1mamsulUp9fX3WZHqKxShePRcKhfDPYrGY53m1WrX1THuUU4sZw0yLxWIog+7ubjOaxhqGsRYKhfYOdGqa/30NK5fr9bqNd3orOTXKtp+kcspnsVCtVtM0LRaL8+fPz7IsqiN85ZVXHnjggc2bN8+bNy/yS2+5XK5UKm2PqvDfXqlU8jxvNBoLFiyIbaYHDx785je/+cd//McxPzMI4VIoFMrlctsrORxMsVis1WpZls2bNy/P86iW63vf+95LL720fv36yBM5nKFhMWM4mHq9Xi6Xu7q6YrvhSdP0nnvuufjii5cuXRr59atcLpfL5alWRk61TalUCltzT09Ps9mM7dJ71VVX7dmzZ+fOnd/5znfK5UhnPbVNVyqVGBKhWCxWq9Vms9nb29tsNqO69L7yyisf+MAHjhw58vOf//yv/uqvYj4vwjSDGHq9UqlkWRbO06jW6ic/+cmHP/zhJEnmz59/4YUXRjvTMNBarZZE8PqvVCplWVYqlbq7u1utVqvVimqtHnjggdtuu+3BBx984okn5s+fH21OlcvlUqlUrVbb/sSxQ56wxHajNus0Go08z5vNZr1ej+rAPve5z/3N3/xN+PNHPvKRW265JdqiqlarlUoldFXbz+owzTRNm81mVJfeVqu1YcOGJ598MvzrF7/4xfe///0xJ/LUTGN4WhDU6/WovjV17Nix88477/Dhw0mSnH766Y899tgZZ5wRbSJXq9VwAY7htifc6jSbzUajEdV97A9+8IN169YlSdLV1bVy5cqvfOUrPT09kT5NKZer1WqpVIrhKbKcIsnzPMuysEdHsphHjx698sord+/efeWVV27duvXyyy//5je/uXz58q997WuLFy+O8CYpedXroUhmmqZp2KMjmemzzz67bt26Q4cO3XjjjX/3d3+3atWqp59+esOGDffcc09vb2+cMw1vhSKZaThJ8zyPZ8d76KGHfv/3f3/hwoWHDh1KkiT84Qtf+MI111wT58OM8EoonrfM4SSNZ6atVusTn/jE3XffHU7Piy666Iknnli4cOHIyMiyZcuinWkMb287g5d9J+ETGW7Xwmbd9uMZHx8/++yzx8bGtm7dumrVqpBT119//YYNG5YuXfqf//mf/f39ES5gbIdULpdDKMcw06effvqCCy4YGBj413/913379iVJ8qUvfemxxx677rrrnnzyyQMHDsR2UhSLxdg26HBI8cz0M5/5zM0333zRRRd9/etff+Mb35gkyb//+79/9KMfveGGG1544YUvfOELsc00nrudVx9SeOsXydOpJUuWjI2N3X777b/7u7+7dOnS/v7+559//vLLLz/77LMfeeSR3/u934ttl4vnbkdO8cvndhQTLZdvvfXWNWvW9Pf379+/PxzY+eef//zzzz/88MNveMMbon3lF2cox3AkZ5xxxvDw8BVXXFEul5977rkkSarV6tDQ0Pr1659++mkDnY0zXbNmzbvf/e61a9dO/eS0007bvn37tddeG85iw5p17X7rrbcuX758+fLlYeMtFApvfvObf/SjHz3++ONvf/vbzVROMcv09vZu2rTptf+c+C1ZsmTJkiWv/efEb/Xq1a/r58Tvf91gy+Xyq6OZTi57SwAAIKcAAOQUAICcAgCQUwAAyCkAADkFACCnAADkFAAAcgoAQE4BAMgpAAA5BQDA/1S2BCdRnucRHk+e57Ed2KsVCgUzNdNTdqDxH5iZTvt4nKSnlELk5/Bsqagsy9I0je3AWq3Wf/zHfyxYsKC3tzfmBSwWi6VSKarTO8/zNE2zLIttrcbHx48ePfobv/Eb5XLU90KlUqlYLEY103CSRrjjvfjii0mSLF68OPKcCudpVEeVpmm0G29PT09/f3/kMw3nqYu4nIpCs9kMjwqazWaEV99Z8BEsFCqVSrj0ViqVeGaapmmr1XKCTK+PK5VKuADHkH3h9AxX3/AHXq9yuRxGWS6XY7gAT4VUs9mMsKhmhUqlEu5jy+Wyh1Un4RyxBDPcpvM8n5iYcN2diUajUSqVenp6sixr+04dHmCMj49nWWam007ker1erVZrtVokTwuyLBsfH0+if6cW831jkiQ9PT2hkmPIqWazOTk5aaYzmWmxWAwbb2zPHeXUKSe8D2q1WgsWLLAa0zA6OpokSWipNE1j2KmzLGs2m/PmzTOdmcy0q6srSZJarZamaXt36nDPU6/XnaQzGWixWOzq6qrX66VSKc/z9j7MCI+jJicnzXTmG2+z2Qxv/SyLnGr/pTfcGz2SP2JBXpf1hfUnTpwol8tdXV1pmsawU4cnUo1Gw0BnMtOurq7JyclKpdJoNKrVant36jDTVqtlptMeaJIkvb294QwNX0Fr7zvccBMb/mym05tpeC4V9rpqtRrDm4HZzvLN6K43SZJGo+HN/bRNTk5OvYIJy9j25/bhFwuMZtrq9frUK5iwku2d6dRHi+k5ceJEvV6f+opSlmXtPUHC40bfr5iJV155ZWJiYmqUNj05xazne8EQuXDDQyeZmJiwCHIKAEBOAQDIKQAA5BQAgJwCAJBTAABy6hQy9XfHYaCYKWZqFsip1+3IkSOVSuXRRx+1FB1jx44dhULhyJEjlqJjXHnllWvWrLHpd5JKpXLXXXdZh0j2zF/7tV/zt5HJqRmZO3fu4ODghz/8YZ+kjrFmzZokSTZt2mQpOsaHPvShXbt2/cM//IOl6BhDQ0NbtmzZt2+fpYhhzxwbG7vssssshZyavnK5vG3btvBJcu/bGfr7+0dGRnbt2nXLLbdYjc6wdu3ajRs3btmy5amnnrIaneHWW28dGBi4+OKLPUiOYc8cHh62Z8qpk/BJ2rNnz65duxYtWrR//34L0hlX35tvvvmmm27atGmTSu4MW7duHRwcPPfcc70h6phb2ccff3xsbGzFihUque02bdo0ODhoz/zlT+mp+T97fHy8r69vJv8JY2Njq1atevHFF32GZu7MM8+M4TC2b9++YMGCz33ucyYyQ2maRjLTLVu2LF68+KKLLjKUGZqYmFi6dGnbD2NsbOy88847ePDgnDlzDGWG3vzmN898z+zv77/99tstZnLKPp3q7e3Np+vw4cMDAwNJktx///0+QCfF6Ojo2NjY0aNH6/V6mqb5r9yePXuSJBkYGLj++uuNY+ZKpdLo6OiRI0d+/vOf1+v1LMt+9TO9+eabkyQZHBw8//zzTWTmuru7R0dHjx49Oj4+Xq/Xf/UDPXHixODgYJIkQ0ND/f39JjJzL7zwwsGDB48ePTo5OdlsNl/XOEZGRsKeecMNN1jJUzqnpq3Vaq1YsSIUwNq1ay1IB9i/f/+55547ODj4/PPPL1myxIJ0gG3btt10000333zzzp07e3t7LUgHuOyyy3bt2jUyMnLbbbdZjfbat2/funXrBgcHDxw4YM+cUrYEr8vHPvaxsbGx0dHRJUuW5HluQTqgjy+99NKBgYGdO3eWy+Usy6xJB/Tx5s2bN27c+OlPfzrx1+R0Sh/v2rVreHh47dq1Nt6275kXX3zx1J5pQeTUdBw5cuSOO+6488479XjH2LFjx969e/fu3Wtf6Bh//ud/PjAwcN9991mKjrF58+bBwUF/oUkke+bY2Jg9U07NSPi1vpUrV1qKjrFmzZq9e/cuX77cUnSMP/uzP7v99tu94+ske/bseec732kdItkzw/sZSyGnZmT16tUWocMS2ddaO4w4tvFiz/zV81V0AAA5BQAgpwAA5BQAgJwCAEBOAQDIKQAAOQUAIKcAAPg/+VvRT5r1hfUWwUAxU8wUOcU0jY6OHjt2LE3TiYkJq/HalUqlOA/s4MGD9Xr9+PHjjUaj0WiYVGecpI1G4+WXX56YmEjT1ILM9pM0zPTEiRP1ev3EiRMm9dpVq1WLIKdi3GumtuZSqdTX12dNpqFQKBSLxfCHth9JkiTFYrFYLOZ5Xq1WbT2ve1v5xf/VfFjMSGZaKpVCGXR3d5vRNDa6qTO0vQOdOkPDH8rlcr1et/G+XnmeT61h22cqpxRAoVAo1Gq1NE2LxeL8+fNjO8L/+q//Ghoa+tSnPrV06dL4N+tqtdr2szpcMGq1Wp7n9Xp9wYIFsS3Uv/zLv9x222333ntvT09P5EVVKBSq1WpY0vZ+tLIsKxaL4VSN8DzdunXr6OjoTTfdFP8jjWKxOFWl7Z1ppVKp1+uVSiVJkq6urqgWKk3TG2+88bLLLnvPe94T+UwrlUpYw5ifQc6aJMjz3CpMW5ZlrVZrfHw8y7I0TaNazFarNTg4+JOf/GT+/PnPPPNMtFffQqEQzuQ5c+bEsFO3Wq00TcO7g1arFdVaHTt27B3veEeSJO9973vvu+++mO80wih7enoqlUrbb3ybzWaz2ZyYmMjzPLbXfE888cTmzZuTJLnnnnve9773RTvT8Ly2VqvF8Lw2z/Nmszk5OdlsNmPbeJMk+eQnP/mP//iPSZI888wzixYtivkmtlAohJN06okycqptwhdrWq1Wo9GIZ6dutVqf/vSn77333vCvF1xwwVe+8pU4i6pYLFar1XCHFMNrtbBTh382m80sy+Jpqfe9730HDhx46aWXkiT5xCc+ccMNN8S5CZZKpVqtFjbrMNm2Py1I0zTLskaj0Ww241mo55577vzzz5/615GRkZUrV8b8GKP0C20/nnCSpmlar9ejSuT777//k5/8ZJIkXV1db3jDG77//e9H+EA0qNVq4W4nhnseOcV/b9ZZlsWzki+++OL73//+ffv2DQ0N3XHHHX/xF3/xmc98ZuHChY8++uiyZcvifJgRXiJEcjx5nofHjfEs0WOPPXbppZcmSRKmGSZ74YUXbt26tb+/P86ZTn3bJpKZhvM0nhuej3/843fffffChQsPHTqUJMmFF164e/fuP/mTP/nbv/3bOCs5nKTxXHdjeydw5MiRK6+8cvfu3Zdccsm3v/3tSy655Jlnnjl06NCXv/zlD37wg3FevMJJqqVOCs/3Ts4nslQq5Xkew4k9Pj7+1re+dWBg4Mc//vGcOXPuuOOOt73tbS+88MLll19+1llnvfDCC0uWLIlwm44wBcJ3bmI4nqeeeurSSy+96KKLhoeHd+3alSTJddddd/HFF1999dVvetObIvwltQi/2VooFEKjRDLTP/3TP7377ru3bNny+c9/vlarJUmyc+fOL37xi0NDQ8eOHRseHjbT17JvhN8XiaSozj777LGxsQcffPCcc8759re/3dfX99Of/vQjH/nIVVddVSqVNm7caOOVU8ym7aa7u3vPnj0rV64sl8v79+8PB/abv/mbP/zhD3/wgx+cccYZTqFZt928613v2rNnz+rVq5NX/VrTunXrnn/++R//+McGOhtn+qlPfeoP//APly9fPvWTarW6ZcuW9773va5zs7TzHn/88YGBgf7+/qmNd86cOdu3b7/22mvf+c53mqmcYrZNtFwO193X+HPi19vb+7/O7v/3c+LX39//v76ljfDhMa/Rq+P41Zykp8qtmiUAAJBTAAByCgBATgEAyCkAAOQUAICcAgCQUwAAcgoAADkFACCnAADkFACAnAIA4H8qW4KTJc/zPM+jOqQsy8KBhT/EqVAoFAoFM33thxQmG/NMi8V479OiXbeYB2qmNl7+7yWN7WoxG2VZlqaplZzJiV0qlaLar9M0zbLMTGdy9S2VSvHs1+Halqap0UxbOEmjmmk4T41m2htvOE8txUnh6dRMNZvNcFY3Go1Wq2VBprFH12q1EC7VajWGPbrZbCZJ0mq1Go2GC/A0VCqVSqWSZVmhUKhUKjHc8LRarTDZZrPpAjwNtVqtUqmEO58YLsCtVis8o202m41Gw4CmccNTq9XK5XKapmGy1kROtfnSm+f5+Pj4vHnzrMY0jI2NpWmapmmxWOzt7U3TtO07dXgo1dXVZTrTnmkIl0ql0tXVled523fq8Bnr6ekxnekNNPyhXq93d3eHooohkScnJ0877TQDmt5MC4VCuK/o6+tL07RcFgNyqt13SFNPmx95xIuh12f9+sLExES4T+rt7W21WjHs1GmahqdTBjqTmYaHUrVaLcuy9s403PPU63UznclAkyTp7u6u1+vlcjnLsva+mg/PjOzhzD8AAAXqSURBVJ2nM59pX19fo9Go1WqWZeb8Zt9Md2rvDmbi2LFjx44dC19qaTQabf/qdxhluPQy7ZlOTk6GS12z2Wz729LwSshL2xmepOEMDSsZw0zDB4xpz/To0aPhDA2vSp0gcqrNLRU+jnJqhmuY/OJbpa/+SdsPiWkLXyKc+s2m9i5p5L9gFb8QLlMV1fZ7nnAAfvtnJtI0DS/7ps5Qiymn6ISdGojW+Pi4RegwUy/7kFMAAHIKAEBOAQAgpwAA5BQAgJwCAJBTAAByCgAAOQUAIKcAAOQUAICcAgBATgEAyCkAADkFACCnAACQUwAAcgoAICplS3CyrF9fsAgGiplipsgppml0dPTYsWONRuPll1+2Gh3g4MGD9Xr9+PHjJ06cqNfrFqQzTtJwhv7sZz9L09SCdMZMwxl67Ngxq4Gc6hzVanX+/PnWoZP09fX19fVZh07yxje+0SJ0GBsvcmoWKxQKSZLUarU8z5MkOe200yYnJ2M7yOPHj8+dOzfqj2C5nCRJsVisVCpTq9rGmRYKhVKpVKlUms1mb29vq9WKbcUmJia6u7tjnmlXV1dYzHK5HJa0vTMNn7FqtZokSXd3d5ZlUS1XeFRWKpWiHWixWKxWq2EZS6VSew81fKLCGVosFnt7eyNcsfhP0nK5XCwWwz8LhUKx6IvUcqqtwnW3Xq93dXU1Go2w3cTj4Ycfvv7662+44YahoaGYl7FWq4WciuHSWygUqtVqmqYh72Jz3XXXPfLII1/96lfPPvvsmK++oajCZt32gwnTzPM8TdOenp7YWuqcc85JkuR73/tebMf2S3tdV1dXqVSK4dJbKpXyPA8fs8nJydieHx88ePC8885bvXr1gw8+GPPGW6lUyuVyuM2QUyfh8hGerDA9eZ43m81msxmeS0W1mAcOHHjXu94V/jwyMvLud7870o9goRCeHNRqtVBUbb+8pWk6OTnZbDZjm+nXv/71j370o0mSzJ8//0c/+lG0V99wxa3ValObdXu1Wq1WqzU5OZmmaWw73h/90R994xvfSJLkggsu+NrXvhbvpaJQmCqqGO40Go1Gq9Wq1+tZlkU101artWzZsvBdrr/+67+++uqrY954y+Vyd3d32584yin+++QJ7w6i2qm/853vXHPNNWmaHj16dNGiRQcOHPjSl750xRVXxLmGU+/7Inm812w28zwPDzPi+Zjdf//9H/vYx97+9rc/99xzSZIsXLjwW9/61jve8Y5oL71JBC+Gpm57Wq1WnudZlsXzpm98fPzaa6/dsWPH1E/e8573/NM//VOcb6/CTKde4Lb9eLIsCy/io9p4X3zxxSuuuOLZZ59NkuS3fuu3fvrTn956661XX311bC8ugjDQ8ObUpVxOxSJN03hukm688ca77757+fLl991332//9m/fe++9Dz300O7duy+88MJvfetbsZ3Y4UlGbPdG4dIbSU6Nj49/4AMf2L1794YNG37nd37nD/7gD/75n//5gx/84KFDh2677bZrrrkmzktvbK8PQh9HklPPPvvs2rVrDx069OUvf/mqq65KkuThhx++9NJLFy5c+NBDD61atSq2mYaTNIaQ+qWoiien/v7v//76669fuHDhAw88cMkll2zYsGHBggVhK965c2d/f3+EORW+OOUKftK2GDrJ4cOHkyS58847m83m6OhokiTDw8N5ng8PDydJcvjwYUs064yMjAwMDIyMjEzNcXR0tNlsDg0NDQwMWJ/ZaGhoaMWKFaOjo1MpEE7ewcHBoaEh6zMbDQwMbNy48cSJE2Hj3bhxY57ne/funTp56WyeTnWy/fv3n3nmmcPDw5s2bbIanWHbtm2bN28eHR1dsmSJ1eiQdwSFQhLZt/SY+ca7cePGbdu2WY1Thy/zAwDIKQAAOQUAIKcAAOQUAAByCgBATgEAyCkAADkFAICcAgCQUwAAcgoAQE4BACCnAABOtrIl6GD9/f3Dw8OrVq2yFB1j1apVw8PD/f39lqJjDA8PW4TO23gXL15sKU4p/w9CPYPmKfbR/QAAAABJRU5ErkJggg==)
10.3 基于循环神经网络的翻译建模 359
显然,根据上下文中提到的“没/吃饭”、“很/饿”,最佳的答案是“吃饭”或者
“吃东西”。也就是,对序列中某个位置的答案进行预测时需要记忆当前时刻之前的
序列信息,因此,循环神经网络应运而生。实际上循环神经网络有着极为广泛的应
用,例如语音识别、语言建模以及即将要介绍的神经机器翻译。
第九章已经对循环神经网络的基本知识进行过介绍,这里再回顾一下。简单来说,
循环神经网络由循环单元组成。对于序列中的任意时刻,都有一个循环单元与之对
应,它会融合当前时刻的输入和上一时刻循环单元的输出,生成当前时刻的输出。这
样每个时刻的信息都会被传递到下一时刻,这也间接达到了记录历史信息的目的。比
如,对于序列 x = {x
1
,...,x
m
},循环神经网络会按顺序输出一个序列 h = {h
1
,...,h
m
},
其中 h
i
表示 i 时刻循环神经网络的输出(通常为一个向量)。
图10.8展示了一个循环神经网络处理序列问题的实例。当前时刻循环单元的输
入由上一个时刻的输出和当前时刻的输入组成,因此也可以理解为,网络当前时刻
计算得到的输出是由之前的序列共同决定的,即网络在不断地传递信息的过程中记
忆了历史信息。以最后一个时刻的循环单元为例,它在对“开始”这个单词的信息
进行处理时,参考了之前所有词(“<sos> 让 我们”)的信息。
RNN Cell RNN Cell RNN Cell RNN Cell
<sos>
让
我们
开始
图 10.8 循环神经网络处理序列的实例
在神经机器翻译里使用循环神经网络也很简单。只需要把源语言句子和目标语
言句子分别看作两个序列,之后使用两个循环神经网络分别对其进行建模。这个过
程如图10.9所示。图中,下半部分是编码器,上半部分是解码器。编码器利用循环神
经网络对源语言序列逐词进行编码处理,同时利用循环单元的记忆能力,不断累积
序列信息,遇到终止符 <eos> 后便得到了包含源语言句子全部信息的表示结果。解
码器利用编码器的输出和起始符 <sos> 开始逐词地进行解码,即逐词翻译,每得到
一个译文单词,便将其作为当前时刻解码器端循环单元的输入,这也是一个典型的
神经语言模型的序列生成过程。解码器通过循环神经网络不断地累积已经得到的译
文的信息,并继续生成下一个单词,直到遇到结束符 <eos>,便得到了最终完整的译
文。
从数学模型上看,神经机器翻译模型与统计机器翻译模型的目标是一样的:在
给定源语言句子 x 的情况下,找出翻译概率最大的目标语言译文 ˆy,其计算如下式: