{
  "appeals": 1,
  "budgets": {
    "baseline_limit_s": 600,
    "verification_timeout_s": 600,
    "worker": {
      "max_completion_tokens": 96000,
      "max_turns": 60,
      "max_wall_seconds": 1500.0,
      "prompt_token_budget": 160000
    }
  },
  "issue": "The next submitter link for manual assessment\nAfter merging the PR #1167 for the issue #1110, we found bugs in the new implementation. The pull request #1167 includes more comments about the details.\r\n\r\n\r\n* The selected next submitter may be such a student who has already received manual grading on a submission other than the latest one. A student should be considered manually graded once any one of his/her submissions has been manually graded. The \"next submitter\" link should not repeat a student who has been manually graded.\r\n* The \"next submitter\" link may repeat the same student who was already being inspected. This happens when the student's earlier submission (but not the latest) has been manually graded even though other students have not received any manual grading.\r\n* When all students have received manual grading, the \"next submitter\" link must stop cycling between students like the old implementation had done. The old code has an if statement with the comment \"There are no more unassessed submitters\". It redirects back to the submission list.\r\n\r\n_Originally posted by @Mankro in https://github.com/apluslms/a-plus/pull/1167#pullrequestreview-1474236719_\r\n            ",
  "source": {
    "image": "docker.io/swerebenchv2/apluslms-a-plus:1199-8afbc2a",
    "platform": "linux/amd64",
    "python": "python",
    "setup": [
      "python -c 'import base64; from pathlib import Path; Path('\"'\"'/tmp/receipts-apluslms__a-plus-1199-test.patch'\"'\"').write_bytes(base64.b64decode('\"'\"'ZGlmZiAtLWdpdCBhL2V4ZXJjaXNlL3Rlc3RzLnB5IGIvZXhlcmNpc2UvdGVzdHMucHkKaW5kZXggNTAxOThlMjkuLmQzMjhhMWNmIDEwMDY0NAotLS0gYS9leGVyY2lzZS90ZXN0cy5weQorKysgYi9leGVyY2lzZS90ZXN0cy5weQpAQCAtMTUxMywzICsxNTEzLDk0IEBAIGNsYXNzIEV4ZXJjaXNlVGVzdChFeGVyY2lzZVRlc3RCYXNlKToKICAgICAgICAgICAgIHNlbGYuZW5yb2xsbWVudF9leGVyY2lzZS5jaGVja19zdWJtaXNzaW9uX2FsbG93ZWQoc2VsZi51c2VyLnVzZXJwcm9maWxlKVswXSwKICAgICAgICAgICAgIEJhc2VFeGVyY2lzZS5TVUJNSVRfU1RBVFVTLkNBTk5PVF9FTlJPTEwsCiAgICAgICAgICkKKworICAgIGRlZiB0ZXN0X25leHRfdW5hc3Nlc3NlZF9zdWJtaXR0ZXJfdmlldyhzZWxmKToKKyAgICAgICAgIyBwYXJzZXMgdGhlIHVzZXIgSUQgZnJvbSB0aGUgVVJMIHJlc3BvbnNlIG9mIE5leHRVbmFzc2Vzc2VkU3VibWl0dGVyVmlldyBmb3IgY29udmVuaWVuY2UuIElmIHRoZSBVUkwgZm9ybWF0IGlzCisgICAgICAgICMgZGlmZmVyZW50IChlLmcuIHJlZGlyZWN0IHdoZW4gYWxsIGhhdmUgYmVlbiBncmFkZWQpIHJldHVybiBqdXN0IHRoZSB1cmwKKyAgICAgICAgZGVmIGdldF91cmxfdXNlcl9pZChhcmdzPScnKToKKyAgICAgICAgICAgIHJlc3BvbnNlID0gc2VsZi5jbGllbnQuZ2V0KAorICAgICAgICAgICAgICAgIGYie2V4ZXJjaXNlLmdldF9hYnNvbHV0ZV91cmwoKX1zdWJtaXR0ZXJzL25leHQtdW5hc3Nlc3NlZC97YXJnc30iKQorICAgICAgICAgICAgdHJ5OgorICAgICAgICAgICAgICAgIHJldHVybiBpbnQocmVzcG9uc2UudXJsLnNwbGl0KCcvc3VibWlzc2lvbnMvJylbMV0uc3BsaXQoJy9pbnNwZWN0LycpWzBdKQorICAgICAgICAgICAgZXhjZXB0IEV4Y2VwdGlvbjoKKyAgICAgICAgICAgICAgICByZXR1cm4gcmVzcG9uc2UudXJsCisKKyAgICAgICAgZGVmIGNyZWF0ZV9zdWJtaXNzaW9uKHVzZXIsIHN1Ym1pc3Npb25fdGltZSk6CisgICAgICAgICAgICBzdWJtaXNzaW9uID0gU3VibWlzc2lvbi5vYmplY3RzLmNyZWF0ZSgKKyAgICAgICAgICAgICAgICBleGVyY2lzZT1leGVyY2lzZSwKKyAgICAgICAgICAgICkKKyAgICAgICAgICAgIHN1Ym1pc3Npb24uc3VibWl0dGVycy5hZGQodXNlcikKKyAgICAgICAgICAgIHN1Ym1pc3Npb24uc3VibWlzc2lvbl90aW1lID0gc3VibWlzc2lvbl90aW1lCisgICAgICAgICAgICBzdWJtaXNzaW9uLnNhdmUoKQorICAgICAgICAgICAgcmV0dXJuIHN1Ym1pc3Npb24KKworICAgICAgICBzZWxmLmNsaWVudC5sb2dpbih1c2VybmFtZT0ic3RhZmYiLCBwYXNzd29yZD0ic3RhZmZQYXNzd29yZCIpCisKKyAgICAgICAgZXhlcmNpc2UgPSBCYXNlRXhlcmNpc2Uub2JqZWN0cy5jcmVhdGUoCisgICAgICAgICAgICBvcmRlcj0xMCwKKyAgICAgICAgICAgIG5hbWU9IlVuYXNzZXNzZWQgRXhlcmNpc2UgU3VibWl0dGVyIFZpZXciLAorICAgICAgICAgICAgY291cnNlX21vZHVsZT1zZWxmLmNvdXJzZV9tb2R1bGUsCisgICAgICAgICAgICBjYXRlZ29yeT1zZWxmLmxlYXJuaW5nX29iamVjdF9jYXRlZ29yeSwKKyAgICAgICAgICAgIHVybD0idW5hc3Nlc3NlZCIsCisgICAgICAgICAgICBtYXhfc3VibWlzc2lvbnM9MSwKKyAgICAgICAgKQorICAgICAgICBleGVyY2lzZS5zYXZlKCkKKworICAgICAgICB1c2VyX3N1Ym1pc3Npb24gPSBjcmVhdGVfc3VibWlzc2lvbihzZWxmLnVzZXIudXNlcnByb2ZpbGUsIHNlbGYueWVzdGVyZGF5KQorCisgICAgICAgIHVzZXIyX3N1Ym1pc3Npb24gPSBjcmVhdGVfc3VibWlzc2lvbihzZWxmLnVzZXIyLnVzZXJwcm9maWxlLCBzZWxmLnRvZGF5KQorCisgICAgICAgICMgdXNlciBzdWJtaXNzaW9uIGRheSBiZWZvcmUgdXNlcjIKKyAgICAgICAgc2VsZi5hc3NlcnRFcXVhbCh1c2VyX3N1Ym1pc3Npb24uaWQsIGdldF91cmxfdXNlcl9pZCgpKQorICAgICAgICBzZWxmLmFzc2VydEVxdWFsKHVzZXIyX3N1Ym1pc3Npb24uaWQsCisgICAgICAgICAgICAgICAgICAgICAgICAgZ2V0X3VybF91c2VyX2lkKGYiP3ByZXY9e3NlbGYudXNlci5pZH0iKSkKKworICAgICAgICB1c2VyMl9zdWJtaXNzaW9uLnN1Ym1pc3Npb25fdGltZSA9IHNlbGYueWVzdGVyZGF5IC0gdGltZWRlbHRhKGRheXM9MSkKKyAgICAgICAgdXNlcjJfc3VibWlzc2lvbi5zYXZlKCkKKworICAgICAgICAjIG5vdyB1c2VyMiBzdWJtaXNzaW9uIGlzIGVhcmxpZXIKKyAgICAgICAgc2VsZi5hc3NlcnRFcXVhbCh1c2VyMl9zdWJtaXNzaW9uLmlkLCBnZXRfdXJsX3VzZXJfaWQoKSkKKworICAgICAgICAjIHdlIHNob3VsZCBub3cgZXhwZWN0IHRvIGdldCB1c2VyMSdzIHN1Ym1pc3Npb24gZmlyc3Qgc2luY2UgdXNlcjJzIGVhcmxpZXIgc3VibWlzc2lvbiBoYXMgYmVlbiBncmFkZWQKKyAgICAgICAgdXNlcjJfc3VibWlzc2lvbi5ncmFkZXIgPSBzZWxmLnRlYWNoZXIudXNlcnByb2ZpbGUKKyAgICAgICAgdXNlcjJfc3VibWlzc2lvbi5zYXZlKCkKKyAgICAgICAgc2VsZi5hc3NlcnRFcXVhbCh1c2VyX3N1Ym1pc3Npb24uaWQsIGdldF91cmxfdXNlcl9pZCgpKQorCisgICAgICAgICMgdGVzdCB3aXRoIHByZXYgcGFyYW1ldGVyCisgICAgICAgIHNlbGYuYXNzZXJ0RXF1YWwodXNlcl9zdWJtaXNzaW9uLmlkLCBnZXRfdXJsX3VzZXJfaWQoZiI/cHJldj17c2VsZi51c2VyMi5pZH0iKSkKKworICAgICAgICAjIHJlbW92ZSBncmFkZXIgZm9yIGZ1cnRoZXIgdGVzdHMKKyAgICAgICAgdXNlcjJfc3VibWlzc2lvbi5ncmFkZXIgPSBOb25lCisgICAgICAgIHVzZXIyX3N1Ym1pc3Npb24uc2F2ZSgpCisKKyAgICAgICAgIyBjcmVhdGUgYSBzdWJtaXNzaW9uIGZvciB1c2VyIHNvIHRoZSBuZXdlc3Qgc3VibWlzc2lvbiB3YXMgbWFkZSBieSB1c2VyIGFnYWluIGFuZCBub3QgdXNlcjIKKyAgICAgICAgdXNlcl9lYXJsaWVyX3N1Ym1pc3Npb24gPSBjcmVhdGVfc3VibWlzc2lvbihzZWxmLnVzZXIudXNlcnByb2ZpbGUsIHNlbGYueWVzdGVyZGF5IC0gdGltZWRlbHRhKGRheXM9MikpCisKKyAgICAgICAgc2VsZi5hc3NlcnRFcXVhbCh1c2VyX3N1Ym1pc3Npb24uaWQsIGdldF91cmxfdXNlcl9pZCgpKQorCisgICAgICAgIHVzZXIyX290aGVyX2V4ZXJjaXNlX3N1Ym1pc3Npb24gID0gU3VibWlzc2lvbi5vYmplY3RzLmNyZWF0ZSgKKyAgICAgICAgICAgIGV4ZXJjaXNlPXNlbGYuZXhlcmNpc2Vfd2l0aF9hdHRhY2htZW50CisgICAgICAgICkKKyAgICAgICAgdXNlcjJfb3RoZXJfZXhlcmNpc2Vfc3VibWlzc2lvbi5zdWJtaXR0ZXJzLmFkZChzZWxmLnVzZXIyLnVzZXJwcm9maWxlKQorICAgICAgICB1c2VyMl9vdGhlcl9leGVyY2lzZV9zdWJtaXNzaW9uLnN1Ym1pc3Npb25fdGltZSA9IHNlbGYueWVzdGVyZGF5IC0gdGltZWRlbHRhKGRheXM9MykKKyAgICAgICAgdXNlcjJfb3RoZXJfZXhlcmNpc2Vfc3VibWlzc2lvbi5zYXZlKCkKKworICAgICAgICAjIGFuIGV2ZW4gZWFybGllciBzdWJtaXNzaW9uIGluIGFub3RoZXIgZXhlcmNpc2UgZG9lc24ndCBtYXR0ZXIKKyAgICAgICAgc2VsZi5hc3NlcnRFcXVhbCh1c2VyX3N1Ym1pc3Npb24uaWQsIGdldF91cmxfdXNlcl9pZCgpKQorCisgICAgICAgICMgd2Ugc2hvdWxkIG5vdyBleHBlY3QgdXNlcjIncyBzdWJtaXNzaW9uIHRvIGJlIHNob3duIGJlY2F1c2UgdXNlcjEncyBsYXRlc3Qgc3VibWlzc2lvbiBoYXMgYmVlbiBncmFkZWQKKyAgICAgICAgdXNlcl9lYXJsaWVyX3N1Ym1pc3Npb24uZ3JhZGVyID0gc2VsZi50ZWFjaGVyLnVzZXJwcm9maWxlCisgICAgICAgIHVzZXJfZWFybGllcl9zdWJtaXNzaW9uLnNhdmUoKQorICAgICAgICBzZWxmLmFzc2VydEVxdWFsKHVzZXIyX3N1Ym1pc3Npb24uaWQsIGdldF91cmxfdXNlcl9pZCgpKQorCisgICAgICAgICMgZ3JhZGluZyBpbiBhbiBvbGRlciBzdWJtaXNzaW9uIChub3QgbGF0ZXN0KSBzaG91bGQgYWxzbyBtZWFuIHVzZXIxIGlzIHNraXBwZWQKKyAgICAgICAgdXNlcl9lYXJsaWVyX3N1Ym1pc3Npb24uZ3JhZGVyID0gc2VsZi50ZWFjaGVyLnVzZXJwcm9maWxlCisgICAgICAgIHVzZXJfZWFybGllcl9zdWJtaXNzaW9uLnNhdmUoKQorICAgICAgICB1c2VyX3N1Ym1pc3Npb24uZ3JhZGVyID0gTm9uZQorICAgICAgICB1c2VyX3N1Ym1pc3Npb24uc2F2ZSgpCisgICAgICAgIHNlbGYuYXNzZXJ0RXF1YWwodXNlcjJfc3VibWlzc2lvbi5pZCwgZ2V0X3VybF91c2VyX2lkKCkpCisKKyAgICAgICAgIyBldmVyeXRoaW5nIGhhcyBiZWVuIGdyYWRlZCAod2Ugc2hvdWxkIGdldCB0aGUgc3VibWlzc2lvbnMgbGlzdCkKKyAgICAgICAgdXNlcjJfc3VibWlzc2lvbi5ncmFkZXIgPSBzZWxmLnRlYWNoZXIudXNlcnByb2ZpbGUKKyAgICAgICAgdXNlcjJfc3VibWlzc2lvbi5zYXZlKCkKKyAgICAgICAgc2VsZi5hc3NlcnRFcXVhbChleGVyY2lzZS5nZXRfc3VibWlzc2lvbl9saXN0X3VybCgpLCBnZXRfdXJsX3VzZXJfaWQoKSkK'\"'\"'))'",
      "git apply /tmp/receipts-apluslms__a-plus-1199-test.patch"
    ],
    "workdir": "/a-plus"
  },
  "spec_paths": [
    "exercise/tests.py"
  ],
  "spec_tests": [
    "exercise/tests.py::ExerciseTest::test_next_unassessed_submitter_view"
  ],
  "suite_scope": "--no-header -rA --tb=line --color=no -p no:cacheprovider -W ignore::DeprecationWarning exercise/tests.py"
}