Editing Arcane University:World space synchronization

Jump to: navigation, search

Warning: You are not logged in. Your IP address will be publicly visible if you make any edits. If you log in or create an account, your edits will be attributed to your username, along with other benefits.

The edit can be undone. Please check the comparison below to verify that this is what you want to do, and then save the changes below to finish undoing the edit.
Latest revision Your text
Line 1: Line 1:
{{Trail}}
 
 
In this article, world space synchronization refers to the process of copying a part of one world space into another, so that the borders match up in both. This is needed for Beyond Skyrim projects whose provinces border each other. Once a project has their border region more or less finalized, it needs to be copied over to the other project so that they can level design their border region while accounting for what's directly on the other side; and once that's done, this border region needs to be copied back the other way.
 
In this article, world space synchronization refers to the process of copying a part of one world space into another, so that the borders match up in both. This is needed for Beyond Skyrim projects whose provinces border each other. Once a project has their border region more or less finalized, it needs to be copied over to the other project so that they can level design their border region while accounting for what's directly on the other side; and once that's done, this border region needs to be copied back the other way.
  
Line 8: Line 7:
 
You need two world spaces, one containing the landscape to copy (henceforth '''origin world space''') and one to copy to ('''destination world space'''). For clarity's sake, this article will assume that they are contained in '''masters''' (.esm) to distinguish them from '''plugins''' (.esp), but the process is the same if they are in plugins. The two world spaces may be part of the same master file, which makes the process a bit simpler, but often will be in separate masters.
 
You need two world spaces, one containing the landscape to copy (henceforth '''origin world space''') and one to copy to ('''destination world space'''). For clarity's sake, this article will assume that they are contained in '''masters''' (.esm) to distinguish them from '''plugins''' (.esp), but the process is the same if they are in plugins. The two world spaces may be part of the same master file, which makes the process a bit simpler, but often will be in separate masters.
  
The copy happens in two passes that are independent and can be done in any order: copying of '''landscape''', and copying of '''references'''. Both methods work on rectangular regions in a world space, so you must figure out what those rectangular regions are (there can be multiple that combine into a more complex shape, even just one cell large if necessary). It helps to have something like a grid map on hand.
+
The copy happens in two passes that are independent and can be done in any order: copying of '''landscape''', and copying of '''references'''. Both methods work on rectangular regions in a world space, so you must figure out what those rectangular regions are (there can be multiple that combine into a more complex shape, even just one cell large if necessary). Write down the cell coordinates of the northeast and southwest corners of every rectangle according to the origin world space's coordinate system. Then, also calculate what those coordinates amount to in [https://www.creationkit.com/index.php?title=Unit '''Units'''] (precise formulas forthcoming). It helps to have a grid map or similar on hand.
  
First, find out the difference between the coordinate systems of the two world spaces. Pick a location that can be pinpointed in both world spaces, then note its cell coordinates in the origin world space, and its cell coordinates in the destination world space. Subtract the X and Y coordinates in the origin world space from the X and Y coordinates in the destination world space, respectively. This will give you the '''offset''', or the number of cells to move the copy by along the X and Y axes. Also calculate the X and Y offsets in units by multiplying them with 4096 (as that is the number of units comprising the edge of a Skyrim world space cell). If the offset is not a clean multiple of a cell, then the method for copying landscape will not work, as the method for that works cell by cell. The method for copying references is unit-based, but will be of little use if the landscape cannot be accurately copied. Lastly, if the sea level is not at the same height in the two world spaces, write down the difference between them in units (this does not have to be a multiple of 4096).
+
Next, find out the difference between the coordinate systems of the two world spaces. Pick a location that can be pinpointed in both world spaces, note its cell coordinates in the origin world spaces, and its cell coordinates in the destination world space. Subtract the X and Y coordinates in the origin world space from the X and Y coordinates in the destination world space, respectively. This will give you the '''offset''', or the number of cells to move the copy by along the X and Y axes. Also calculate the X and Y offsets in units by multiplying them with 4096 (as that is the number of units comprising the edge of a Skyrim world space cell). If the offset is not a clean multiple of a cell, then the method for copying landscape will not work, as the method for that works cell by cell. The method for copying references is unit-based, but will be of little use if the landscape cannot be accurately copied. Lastly, if the sea level is not at the same height in the two world spaces, write down the difference between them in units (this does not have to be a multiple of 4096).
 
 
Write down the cell coordinates of the northeast and southwest corners of every rectangle according to the origin world space's coordinate system. Add the offsets onto this to get the coordinates in the destination world space.
 
 
 
Lastly, also calculate what those coordinates amount to in [https://www.creationkit.com/index.php?title=Unit '''Units''']. You will need the southwest corner of the southwest cell, which you can find by multiplying the X and Y cell coordinates by 4096. You will also need the northeast corner of the northeast cell, for which you must multiply the X and Y cell coordinates by 4096 and add an extra 4096.
 
 
 
This spreadsheet template will help you organize the information, calculate most of it for you, and present it in the form that's required in the next steps. [[File:World space synchronization template.zip]]
 
  
 
If the two world spaces belong to different masters, and the destination world space's file is not a master of the origin world space's file, then the copied area must be free of land textures and references whose base objects come from the origin world space's file. This can be resolved in advance (such as by re-implementing any non-vanilla assets in a shared master, and switching all references to use those shared base objects; for Beyond Skyrim, this is BSAssets) but can also be arranged at the end of the process; this article will show how to contain the copy in a patch plugin which can be tweaked before being sent off to be merged into the destination world space's master. If it is too difficult to integrate land textures, these can optionally not be copied as detailed further below.
 
If the two world spaces belong to different masters, and the destination world space's file is not a master of the origin world space's file, then the copied area must be free of land textures and references whose base objects come from the origin world space's file. This can be resolved in advance (such as by re-implementing any non-vanilla assets in a shared master, and switching all references to use those shared base objects; for Beyond Skyrim, this is BSAssets) but can also be arranged at the end of the process; this article will show how to contain the copy in a patch plugin which can be tweaked before being sent off to be merged into the destination world space's master. If it is too difficult to integrate land textures, these can optionally not be copied as detailed further below.
Line 39: Line 32:
 
=== Landscape ===
 
=== Landscape ===
  
Here, landscape refers to the LAND record in each cell, containing land height (i.e. heightmap data) and the land textures used (which refer to LTEX base objects). In xEdit, you need to disable the "Simple records" option or you will not be able to copy landscape. Find the options menu by clicking on the three horizontal bars, then "Options"; then under the General tab, make sure "Simple records LAND, NAVI, NAVM, CELL, WRLD (requires restart)" is unticked.
+
Here, landscape refers to the LAND record in each cell, containing land height (i.e. heightmap data) and the land textures used (which refer to LTEX base objects).
  
 
If you wish to copy land textures, and some of them still come from the origin world space's file, then you must temporarily make the origin file a master of the destination file for the copying to work; the copying will also not happen in a patch plugin, but is applied directly on the destination file. However, a later step will extract the edits into a patch plugin after all.
 
If you wish to copy land textures, and some of them still come from the origin world space's file, then you must temporarily make the origin file a master of the destination file for the copying to work; the copying will also not happen in a patch plugin, but is applied directly on the destination file. However, a later step will extract the edits into a patch plugin after all.
  
Right click anywhere and choose "Apply script". Choose the built-in script "Worldspace copy landscape area to another worldspace". If you want to copy land textures, edit the script so that <code>bCopyLayers = True</code>. You can do this directly in xEdit in the "Apply Script" dialogue by simply typing into the script after you have selected it in the drop-down. When you see "The script has been modified. Do you want to save it?", answer Yes.
+
Apply a script on the origin world space (right click on the world space in the origin file, then Apply script). Choose the built-in script "Worldspace copy landscape area to another worldspace". If you want to copy land textures, edit the script so that <code>bCopyLayers = True</code>.
  
 
A pop-up window appears. Configure the two world spaces; enter the cell coordinates of the southwest corner of the current rectangle, and sizes of the rectangle along both axes, which is the difference between the two X and Y coordinates respectively. A rectangle comprised of one cell will have sizes 1 and 1; a square of 2x2 cells will be 2 and 2, and so on. Then enter the southwest cell coordinates in the destination world space by summing the cell offsets with the origin world space's coordinates. You must leave "Create patch plugin" unchecked ''unless you have no problematic land textures or forego copying them entirely'', and if you check it, every rectangle will go in its own patch plugin which you will want to merge afterward.
 
A pop-up window appears. Configure the two world spaces; enter the cell coordinates of the southwest corner of the current rectangle, and sizes of the rectangle along both axes, which is the difference between the two X and Y coordinates respectively. A rectangle comprised of one cell will have sizes 1 and 1; a square of 2x2 cells will be 2 and 2, and so on. Then enter the southwest cell coordinates in the destination world space by summing the cell offsets with the origin world space's coordinates. You must leave "Create patch plugin" unchecked ''unless you have no problematic land textures or forego copying them entirely'', and if you check it, every rectangle will go in its own patch plugin which you will want to merge afterward.
  
Then allow the script to run. LAND records are heavy and so this will take some time. For a large rectangle, xEdit may run out of memory, so consider using the 64bit version. After a rectangle has been copied, you can save to free the memory back up. If you see an error like "Load order FileID [06] can not be mapped to file FileID for file", then that means some land textures were used that are not from a master of the destination esm, and you may have to apply the aforementioned workaround.
+
Then allow the script to run. LAND records are heavy and so this will take some time. For a large rectangle, xEdit may run out of memory, so consider using the 64bit version. After a rectangle has been copied, you can save to free the memory back up.
 
 
Once you are done with all rectangles, and you had "Create patch plugin" checked, then you have a bunch of different plugins that you will now have to merge together. If you didn't have "Create patch plugin" checked, then the destination .esm was edited directly and you will now probably want to extract those edits into a patch plugin after all. Save and close xEdit just in case so you can make a back-up of your destination esm. Put a fresh copy of your destination esm in your data folder, one free of any land copying. In xEdit, load only this fresh esm. Once loaded, right click it, select "Compare to..." and select your modified esm. This will make a new load order in xEdit where the modified esm appears to override the records of the fresh one. Right click anywhere and select "Apply filter"; in the pop-up, check "by Base record signature" and check "LAND". Also make sure "by conflict status overall" is still checked, and deselect the top three options. Also check "Flatten Blocks/Sub-Blocks" and "Flatten Cell Children" further below. Run the filter and wait a (long) while. After it's done, only the changed LAND records should still be visible in xEdit. In your modified esm, Alt+Click the expand icon next to the destination world space to instantly open all cells, select one LAND record, press Ctrl+A to select all other LAND records as well, right click and select "Copy as override into...". Make it a new plugin. This plugin now contains all the results of the process, and you can test it alongside the fresh esm.
 
  
N.B.: The "Worldspace copy landscape area to another worldspace" xEdit script only takes into account the plugin file (.esm or .esp) which the origin world space is from; it does not incorporate overrides to the landscape from subsequent plugins in the load order. If you have such edits, you will therefore have to merge them into master first. If you are copying from a vanilla world space (from Skyrim.esm) and need to incorporate edits, then lord help you, because xEdit will not let you modify Skyrim.esm. You will need to create a copy of Skyrim.esm and rename it to something else, for example Skyrim2.esm; launch xEdit with the launch parameters <code>-IKnowWhatImDoing</code> and <code>-AllowMasterFilesEdit</code>; change the file header of the editing plugin so that its master is Skyrim2.esm instead of Skyrim.esm; merge the edits (for instance, right click and "copy as override (with overwriting)", and if that causes issues, clicking and dragging from right to left, one record at a time); save and quit xEdit, and rename Skyrim2.esm back to Skyrim.esm, replacing the original. You can then run the "Worldspace copy landscape area to another worldspace" script. If you are copying from the Tamriel world space, then you have an additional problem in that its FOrmID is 0000003C and FormIDs lower than xx000800 are reserved by the game engine and thus also treated specially by xEdit, so you may have to "Change FormID" on the world space before merging in the edits, and change it back afterwards.
+
Once you are done with all rectangles, and if you didn't have "Create patch plugin" checked, save and close xEdit just in case so you can make a back-up of your destination esm. Put a fresh copy of your destination esm in your data folder, one free of any land copying. In xEdit, load only this fresh esm. Once loaded, right click it, select "Compare to..." and select your modified esm. This will make a new load order in xEdit where the modified esm appears to override the records of the fresh one. Right click anywhere and select "Apply filter"; in the pop-up, check "by Base record signature" and check "LAND". Also make sure "by conflict status overall" is still checked, and deselect the top three options. Also check "Flatten Blocks/Sub-Blocks" and "Flatten Cell Children" further below. Run the filter and wait a (long) while. After it's done, only the changed LAND records should still be visible in xEdit. In your modified esm, Alt+Click the expand icon next to the destination world space to instantly open all cells, select one LAND record, press Ctrl+A to select all other LAND records as well, right click and select "Copy as override into...". Make it a new plugin. This plugin now contains all the results of the process, and you can test it alongside the fresh esm.
  
 
=== References ===
 
=== References ===
Line 67: Line 58:
 
function Filter(e: IInterface): Boolean;
 
function Filter(e: IInterface): Boolean;
 
var
 
var
 +
  stat: IInterface;
 
   xp, yp: float;
 
   xp, yp: float;
 
begin
 
begin
Line 138: Line 130:
 
Alt+Click the expand icon next to the world space, select one reference, press Ctrl+Alt to select all references in the world space, and "Copy as override into" a new plugin. This may take a while. If you have more rectangles, then right click and Remove filter, and run the script again. This time, "copy as override" into the plugin you previously created for convenience's sake.
 
Alt+Click the expand icon next to the world space, select one reference, press Ctrl+Alt to select all references in the world space, and "Copy as override into" a new plugin. This may take a while. If you have more rectangles, then right click and Remove filter, and run the script again. This time, "copy as override" into the plugin you previously created for convenience's sake.
  
'''This paragraph pertains to copying references for LOD only'''  
+
'''This paragraph pertains to copying references for LOD only.''' Areas that are far enough away from the playable border to never be in the loaded area, may still be visible from a distance and thus will have to be copied over to be included in LOD that is generated for the destination world space. Here, you may like to exclude references that won't be visible in LOD. Once you have your .esp from the previous step, apply a Filter for Base record signature "Tree" (shrubs and other small flora will also be lumped in). Copy these as override from your previous .esp into a new one. Remove the filter. Then do another filter by running the below script by Zilav, which retains only references whose base object has LOD meshes configured. Copy all remaining references from the same previous .esp into the same second .esp you created. You may now discard the first .esp and continue with the second one.
 
 
Areas that are far enough away from the playable border to never be in the loaded area, may still be visible from a distance and thus will have to be copied over to be included in LOD that is generated for the destination world space. Here, you may like to exclude references that won't be visible in LOD. Once you have your .esp from the previous step, apply a Filter for Base record signature "Tree" (shrubs and other small flora will also be lumped in). Copy these as override from your previous .esp into a new one. Remove the filter. Then do another filter by running the below script by Zilav, which retains only references whose base object has LOD meshes configured (the built-in "visible when distant" filter options are not effective). Copy all remaining references from the same previous .esp into the same second .esp you created. You may now discard the first .esp and continue with the second one.
 
  
 
  <nowiki>unit ApplyCustomScripted;
 
  <nowiki>unit ApplyCustomScripted;
Line 198: Line 188:
  
 
end.</nowiki>
 
end.</nowiki>
 
'''Filtering Tree References for LOD'''
 
 
After copying all tree references for LOD into your esp, it is ideal to filter these references so that only those with LOD billboards are preserved. The easiest way to obtain a list of which tree records lack billboards is to run xLODGen with your LOD reference plugin loaded. Select the worldspace where the references are added, and only tick the box for "Tree LOD". Generate LOD. The program's log will proceed to list every tree record in the worldspace both with and without LOD. Copy the list of records that lack billboards and save it in a new file. You can now open your LOD reference plugin in xEdit and remove the record types you've listed from your esp. Depending on the area you're copying from, you can reduce the esp's references between 50% and 90%. As references without LOD billboards will not be seen in LOD only cells, these references do nothing but bloat the esp needlessly.
 
 
  
 
Once your patch .esp has all the needed references from all rectangles, save and close xEdit and make a backup. Load it again, apply a filter that only does "Flatten blocks/sub-blocks" and "flatten cell children", select all references in the plugin using the Alt+Click/Ctrl+Click method, right click, and "Change FormID". Select the current plugin. This will change all the overrides into new references, but it will take some time.
 
Once your patch .esp has all the needed references from all rectangles, save and close xEdit and make a backup. Load it again, apply a filter that only does "Flatten blocks/sub-blocks" and "flatten cell children", select all references in the plugin using the Alt+Click/Ctrl+Click method, right click, and "Change FormID". Select the current plugin. This will change all the overrides into new references, but it will take some time.
Line 210: Line 195:
 
After the script is done, your plugin should be ready, unless there were references whose base object comes from the origin esm. In that case, your plugin will depend on both esms and must be tweaked and cleaned before it can be merged into the destination esm.
 
After the script is done, your plugin should be ready, unless there were references whose base object comes from the origin esm. In that case, your plugin will depend on both esms and must be tweaked and cleaned before it can be merged into the destination esm.
  
=== Final checks and extras ===
+
=== Final checks ===
  
You independently created a patch plugin for landscape and another one for references; you may as well merge them together. If base objects from the origin esm are still in use, you can detect them by right clicking your plugin, and applying the built-in script "Report masters", then selecting your origin esm. This will list all such references and what base objects they use, and also LAND records where an incompatible land texture is used (look for mentions of "/ Layers" in the xEdit output); but frustratingly, it will not list the land texture itself.
+
You independently created a path plugin for landscape and another one for references; you may as well merge them together. If base objects from the origin esm are still in use, you can detect them by right clicking your plugin, and applying the script "Report masters", then selecting your origin esm. This will list all such references and what base objects they use, and also LAND records where an incompatible land texture is used; but frustratingly, it will not list the land texture itself.
  
To replace base objects, you can still use the script shown above in the Preparation section. After everything is replaced as needed, right click on your patch plugin and select "Clean masters". If nothing from the origin esm is used anymore, then it will now be removed as master and the plugin can be merged.
+
To replace base objects, you can still use the script shown above in the Preparation section.
  
 
Among the references, remove any map markers, doors and other gameplay-related content that doesn't belong. Most of these can be easily found in the persistent reference cell.
 
Among the references, remove any map markers, doors and other gameplay-related content that doesn't belong. Most of these can be easily found in the persistent reference cell.
Line 309: Line 294:
  
 
end.</nowiki>
 
end.</nowiki>
 
If you need to batch mark as delete a selection of references (for instance, to clear the land and make room for a copy coming in), select the references in xEdit and apply this custom script:
 
 
<nowiki>{
 
  Delete references, a simple edit of the vanilla "undelete and disable references" script.
 
  Does not require applied filter.
 
}
 
unit UserScript;
 
 
 
 
function Process(e: IInterface): integer;
 
var
 
  Sig: string;
 
  xesp: IInterface;
 
begin
 
  Result := 0;
 
 
 
  if not (IsEditable(e)) then
 
    Exit;
 
 
 
  Sig := Signature(e);
 
 
  if
 
    (Sig <> 'REFR') and
 
    (Sig <> 'PGRE') and
 
    (Sig <> 'PMIS') and
 
    (Sig <> 'ACHR') and
 
    (Sig <> 'ACRE') and
 
    (Sig <> 'NAVM') and
 
    (Sig <> 'PARW') and // Skyrim
 
    (Sig <> 'PBAR') and // Skyrim
 
    (Sig <> 'PBEA') and // Skyrim
 
    (Sig <> 'PCON') and // Skyrim
 
    (Sig <> 'PFLA') and // Skyrim
 
    (Sig <> 'PHZD')    // Skyrim
 
  then
 
    Exit;
 
 
 
 
  AddMessage('Deleting: ' + Name(e));
 
 
 
  // delete
 
  SetIsDeleted(e, True);
 
 
 
end;
 
 
end.
 
</nowiki>
 
 
[[Category:Arcane University-Implementation]] [[Category:Arcane University-Level Design]] [[Category:Arcane University-Project Management]]
 

Please note that all contributions to Beyond Skyrim are considered to be released under the Creative Commons Attribution-ShareAlike (see Beyond Skyrim Wiki:Copyrights for details). If you do not want your writing to be edited mercilessly and redistributed at will, then do not submit it here.
You are also promising us that you wrote this yourself, or copied it from a public domain or similar free resource. Do not submit copyrighted work without permission!

To edit this page, please answer the question that appears below (more info):

Cancel | Editing help (opens in new window)

Template used on this page: