DotA Custom-Map Playability

OpenRealm cannot yet play Defense of the Ancients (custom A00Y abilities, HUD draw). The load path on feature/dota-jass-followup parses the 4.1 MiB CR-only Vexorian script and enters main(): identifier scan stops at quotes, fourccs, and 0x/$ hex so 'Hpal'or / 851983then / I2R(0x540BE3FF) tokenize; constant function is accepted; remaining used natives are registered.

This document is the evidence and work order for that map. Reaching com_frame_limit later will still not mean the game is playable.

Contract

DotA 6.83d is a TFT custom scenario (W3I format 25, gameDataSet Custom). The PMV 1.42 EN file is a protected Chinese AI modification of IceFrog 6.83d. The script identifies itself as 6.83d.1.5. Retail Warcraft III loads it because Storm still hashes original member names; the (listfile) is fake binary and must not be trusted.

Authoritative inner members (hash lookup, not the listfile):

Member	Size	Role
`scripts\war3map.j`	4,164,351	Map script. There is no `war3map.j`.
`war3map.w3a`	1,031,633	1,364 original-table ability modifications. Custom table is empty.
`Units\CampaignUnitFunc.txt` / `CampaignUnitStrings.txt`	53,399 / 114,187	810+ map-local unit rows (heroes such as `Harf` Omniknight).
`Units\CampaignAbilityFunc.txt` / `CampaignAbilityStrings.txt`	134,854 / 564,817	~490 map-local ability rows.
`Units\ItemFunc.txt` / `ItemStrings.txt` / `ItemAbilityFunc.txt`	75,575 / 166,832 / 126,167	1,000+ item rows.
`war3mapMisc.txt`	1,761	DotA constants: `MaxHeroLevel=25`, custom `NeedHeroXP`, magic armor table, `MaxUnitSpeed=522`.
`war3map.w3e` / `war3map.doo` / `war3map.wpm`	terrain / doodads / pathing	Present. `war3mapUnits.doo`, `war3map.w3u`, and `war3map.w3t` are absent; units and items are script- and INI-authored.

The archive is MPQ v1 at file offset 512 (HM3W user data). Header wSectorSizeShift is 15 (16 MiB sectors). Storm's sector size is 512 << wSectorSizeShift (max shift 15). OpenRealm must honor that value; clamping anything above 64 KiB down to 4096 makes SectorTableLooksValid reject every multi-kilobyte compressed member. 1,299 files exist; 1,296 of them are compressed+encrypted. One file uses PKWARE implode ((listfile)).

Data Flow

loose Maps/*.w3x
  -> CM_LoadMap opens the map as a nested MPQ and mounts it via FS_SetPriorityArchive
  -> CM_ReadUnits / CM_ReadItems / CM_ReadAbilities parse war3map.w3u / w3t / w3a into MAPINFO
  -> CM_ReadMapScript tries war3map.j, then scripts\war3map.j (case-insensitive)
  -> miss logs and leaves mapscript NULL; jass_dobuffer refuses NULL
  -> G_ApplyMapGameDataSet reloads sheets; G_ReadGameDataFile sees map members first
  -> G_SetMapUnitOverrides / G_SetMapAbilityOverrides
  -> G_SpawnEntities skips jass_dobuffer when mapscript is NULL
  -> G_StartScripts calls main()

Gameplay sheets (Units\CampaignUnitFunc.txt, war3mapMisc.txt) go through G_ReadGameDataFile: current map archive, then Custom_V1\<path>, then the base MPQ path. Renderer map-asset scope still covers models and textures only.

Diagnostic Workflow

Bounded dedicated run (isolated home, unique UDP port):

XDG_DATA_HOME=/tmp/dota-home \
build/bin/openwarcraft3 \
  -data 'data/Warcraft III' -tft \
  +dedicated 1 +set game_port 28301 \
  +set com_fast_forward 1 +set vid_hidden 1 \
  +map 'Maps/DotA v6.83dAI PMV 1.42 EN.w3x' \
  +com_frame_limit 600

Same smoke path through the campaign map auditor (--loose-map skips the War3.mpq / War3x.mpq campaign walk and still records SIGSEGV / timeout):

python3 tools/wc3_map_audit.py \
  --data 'data/Warcraft III' \
  --binary build/bin/openwarcraft3 \
  --mpqtool build/bin/mpqtool \
  --jobs 1 --frames 10 --timeout 60 \
  --loose-map 'data/Warcraft III/Maps/DotA v6.83dAI PMV 1.42 EN.w3x'

make audit-wc3-maps WC3_AUDIT_ARGS="--jobs 1 --frames 10 --timeout 60 --loose-map 'data/Warcraft III/Maps/DotA v6.83dAI PMV 1.42 EN.w3x'"

Observed on 6f1057f1:

CM_ReadDoodads: missing war3map.doo and CM_ReadUnitDoodads: missing war3mapUnits.doo. war3map.doo is in the hash table; on builds that still clamp sector size > 65536 down to 4096 (common/mpq.c before #435), open fails. Small members such as war3map.w3i still open (one sector under both sizes). After #435 the authored 16 MiB sector size is honored.
Overlay spam: Custom_V1\Units\*.txt and Custom_V1\war3mapMisc.txt miss (expected). After #432 the unprefixed path consults the mounted map archive before base TFT, so map Units\ and war3mapMisc.txt win.
EXC_BAD_ACCESS in jass_remove_comments at jdo.c:1980 with buf == NULL, because CM_ReadMapScript stored a null war3map.j.

Hash-known names that mpqtool cat cannot open until sector size is honored include scripts\war3map.j, war3map.w3e, war3map.w3a, and war3map.doo.

JASS Demand (from `scripts\war3map.j`)

The compiled script is 132,869 lines and 5,708 functions (Vexorian-style obfuscated names, original native names preserved). config and main exist.

Against games/warcraft-3/game/common.txt (1,549 natives) and api_module.c (965 registered callbacks):

Bucket	Natives used by this map	Identifier refs
Registered	388	majority of call sites
Unregistered	137	12,465

Unregistered groups, by identifier count:

Group	Natives	Refs	Why it matters
Hashtables (`InitHashtable`, `GetHandleId`, `Save`/`Load`/`HaveSaved`/`FlushChild`/`RemoveSaved*`)	53	8,692	DotA 6.83d's primary data model (YDWE / patch 1.24). Runtime natives + save/load registry in `api_hashtable.h` / `g_hashtable.c` (#437).
`GetObjectName`	1	833	Ability/item/unit tooltips and chat.
Multiboard	17	1,022	Scoreboard / KDA / player list. DotA surface implemented (#434); HUD draw deferred — multiboard-and-texttag.md.
`GetEventDamageSource`	1	213	Kill credit, lifesteal, on-hit scripts.
`SetUnitAbilityLevel` / `IncUnitAbilityLevel` / `UnitDamageTarget` / `GetUnitCurrentOrder` / `GetUnitLevel` / type flags	10	510	Skill ranks and spell scripts.
Shop (`GetSoldUnit`, `GetBuyingUnit`, `GetSellingUnit`, stock add/remove)	6	192	Side shop / secret shop / tavern.
TextTag	10	225	Gold/last-hit floating text. DotA surface implemented (#434); client draw deferred — multiboard-and-texttag.md.
`GetHeroStr` / `GetHeroAgi` / `GetHeroInt`	3	123	Attribute-scaling spells.
Item user-data / visibility / pawnable	10	239	Courier and stash.
Lightning / image / ubersplat	13	90	Skill presentation.
`StringHash` / `StringCase` / `StringLength`	3	285	Hashtable keys.

A gamecache G global remains, and campaign game-cache natives are already registered. This map's live state is hashtables, not game cache.

Registered-but-stub natives this script still calls include SetPlayerAbilityAvailable (650), SetUnitPathing, UnitApplyTimedLife, PauseUnit, SetUnitInvulnerable, and most sound-parameter setters. Treat those as partial, not as coverage.

How Far

Load no longer crashes. The 4.1 MiB CR-only mapscript parses and G_StartScripts calls main() (#441). Lexer: stop identifiers at "/', scan 0x/$ hex and decimals separately so 851983then / 'Ucrl'or( split, constant function parses as a function. parser_line counts \r. Used natives that were still unregistered (SetCreepCampFilterState, GetLocationZ, CopySaveGame, terrain pathing, DestroyForce, TerrainDeformRipple) are registered. main() then spends a long time filling arrays (jass_set_array_value) before returning to the frame loop — +com_frame_limit does not tick until main() returns. Map AbilityData rows with custom rawcodes can inherit a registered mechanic through W3A DataA–I fields and AbilityMetaData.slk's useSpecific list. Shared fields such as Ocl1 also belong to Healing Wave and Forked Lightning, so a map-authored Order from CampaignAbilityFunc.txt selects the matching mechanic; ambiguous rows without that identity are logged and left unresolved. The A00Y fixture exercises Chain Lightning with authored rank-five values and the normal spell cast path. This does not implement custom JASS-triggered ability behavior or HUD drawing.

Suggested order (GitHub #431 and children):

Honor MPQ sector sizes above 64 KiB so protected maps open (#435).
Read scripts\war3map.j when war3map.j is absent; refuse null mapscripts instead of crashing (#433).
Implement the hashtable native family (#437) — runtime + save/load registry done; see below.
Resolve map-archive Units\*.txt, war3mapMisc.txt, and war3map.w3a through the existing sheet/object-data path (#432 — map FS priority + w3a parse/apply for DataA–I and common leveled fields; full AbilityMetaData coverage remains follow-up).
Shop, damage-source, hero-attribute, texttag, and multiboard natives (#434, #436). Issue #436 registers GetEventDamageSource, Set/IncUnitAbilityLevel, UnitDamageTarget, GetHeroStr/Agi/Int, shop stock/sell context, item user-data natives, StringHash/Case/Length, and minimal lightning/image/ubersplat handles. SetPlayerAbilityAvailable now stores per-player disabled ability rawcodes and gates authored command-card visibility plus shared command/order/validation/execute/autocast activation.
Custom ability mechanics (#448 — custom rawcode → base mechanic inference, one-based DataA–I application, and map-authored ranks are implemented for AbilityData-backed mechanics; custom JASS-triggered behavior remains follow-up).

Parsing 4.1 MiB / 5,708 functions is unproven. Budget time for VM capacity after the mapscript actually loads.

Known Pitfalls

The fake (listfile) is implode-compressed garbage. mpqtool ls prints it and looks like a corrupt archive. Hash lookup of known WC3 names still works.
CM_ReadHeightmap is unused; terrain open failures are easy to miss because doodad errors print first.
Overlay logs for missing Custom_V1\... are expected for this map and are not the same as "the map has no CampaignUnitFunc". The map has one; after #432 the unprefixed lookup reads it from the mounted map archive.
Do not treat a later completed audit row as playable DotA. Spawn, shops, items, and win conditions are all script-driven.

Verification

There is no automated DotA scenario yet. After the load bugs, a dedicated run must get past jass_dobuffer and main() without SIGSEGV or unimplemented native: InitHashtable. Object-data work needs tests that a map-scoped Units\CampaignUnitFunc.txt and war3map.w3a row win over base TFT data. Hashtable natives need handle-lifetime tests, including FlushChildHashtable and typed Save*Handle / Load*Handle.

Hashtable save/load

Runtime natives live in games/warcraft-3/game/api/api_hashtable.h and are registered from api_module.c. Tables are host-owned registry slots on level.hashtables[MAX_HASHTABLES] (G_AllocHashtable / jass_pushlighthandle), not VM-owned API_ALLOC blobs. InitHashtable identity is the stable slot ordinal (via G_SaveJassHandle("hashtable", …)); JSVM snapshot version 4 is unchanged because the host codec relocates the light handle.

Each HT_HANDLE entry stores the JASS type string (unit, item, player, …) alongside the pointer so save/load can call G_SaveJassHandle / G_LoadJassHandle. Scalars persist by value. Nested types without a host domain (location, lightning, image, ubersplat, and other VM-owned payloads) log once to stderr and restore as null rather than writing a raw address. Stale/removed units become null, matching the group/unit handle rule. Save format version 31 introduced the registry inuse bits to the level field stream plus a per-slot entry payload after edicts, so nested unit/item handles resolve against restored inuse bits (production SV_Map runs main() before ReadGame). See Save/Load.